2024-04-25T11:26:05.480 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T11:26:05.480 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi --archive /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327 --owner scheduled_leonidus@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/misc}} --verbose -- /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327/orig.config.yaml 2024-04-25T11:26:05.539 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T11:26:05.752 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/misc}} email: null first_in_suite: false job_id: '7673327' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: df40303764f976ac2e973c1ef2b40d1ebbda473b owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2866 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: df40303764f976ac2e973c1ef2b40d1ebbda473b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGISUBihUgZsFxZud9ytPOEp95MyZ0z3iUwK7r2LTaPv9wcVxyoCXKiMLbb7BMbL2owZiIAfB/BhhitLQhbefh0= smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPo3TArzamFKHNhq3G02/mvY2LXS2S1nf+/IN1v8ExnA0UOnS1C03TIXrv4muGJVaEks6Iei29SeYmeQ4XuSd4M= smithi111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKMmefO1xnIngfSC8rDW1pi8MU52/VOEWpehgYjOL2qp7AM6EDBE6uOSR0LEhAOLAmDpC7BbPcx2GVHaSmI305U= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_11:13:36 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T11:26:05.752 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa; will attempt to use it 2024-04-25T11:26:05.753 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa/tasks 2024-04-25T11:26:05.753 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T11:26:05.755 INFO:teuthology.task.internal:Checking packages... 2024-04-25T11:26:05.777 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T11:26:05.777 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T11:26:05.778 INFO:teuthology.packaging:ref: None 2024-04-25T11:26:05.778 INFO:teuthology.packaging:tag: None 2024-04-25T11:26:05.778 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T11:26:05.778 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:26:05.778 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T11:26:06.053 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T11:26:06.055 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T11:26:06.062 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T11:26:06.062 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T11:26:06.075 INFO:teuthology.task.internal:Saving configuration 2024-04-25T11:26:06.097 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T11:26:06.106 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T11:26:06.126 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:20:29.871160', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGISUBihUgZsFxZud9ytPOEp95MyZ0z3iUwK7r2LTaPv9wcVxyoCXKiMLbb7BMbL2owZiIAfB/BhhitLQhbefh0='} 2024-04-25T11:26:06.146 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:20:29.869416', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPo3TArzamFKHNhq3G02/mvY2LXS2S1nf+/IN1v8ExnA0UOnS1C03TIXrv4muGJVaEks6Iei29SeYmeQ4XuSd4M='} 2024-04-25T11:26:06.167 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:20:29.867476', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKMmefO1xnIngfSC8rDW1pi8MU52/VOEWpehgYjOL2qp7AM6EDBE6uOSR0LEhAOLAmDpC7BbPcx2GVHaSmI305U='} 2024-04-25T11:26:06.167 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T11:26:06.176 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T11:26:06.176 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T11:26:06.176 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T11:26:06.176 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T11:26:06.277 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f25462a60d0>, signals=[15]) 2024-04-25T11:26:06.277 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T11:26:06.286 INFO:teuthology.task.internal:Opening connections... 2024-04-25T11:26:06.286 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T11:26:06.288 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:26:06.363 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T11:26:06.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:26:06.436 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2024-04-25T11:26:06.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:26:06.509 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T11:26:06.516 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-04-25T11:26:06.534 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-04-25T11:26:06.534 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-04-25T11:26:06.591 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2024-04-25T11:26:06.591 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2024-04-25T11:26:06.591 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2024-04-25T11:26:06.591 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2024-04-25T11:26:06.591 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2024-04-25T11:26:06.591 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2024-04-25T11:26:06.591 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T11:26:06.591 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2024-04-25T11:26:06.591 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2024-04-25T11:26:06.591 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T11:26:06.591 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2024-04-25T11:26:06.591 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T11:26:06.592 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T11:26:06.592 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T11:26:06.592 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-04-25T11:26:06.616 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-04-25T11:26:06.634 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-04-25T11:26:06.635 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-04-25T11:26:06.692 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-04-25T11:26:06.692 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-04-25T11:26:06.692 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-04-25T11:26:06.692 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-04-25T11:26:06.692 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-04-25T11:26:06.692 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-04-25T11:26:06.693 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T11:26:06.693 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-04-25T11:26:06.693 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-04-25T11:26:06.693 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T11:26:06.693 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-04-25T11:26:06.693 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T11:26:06.693 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T11:26:06.693 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T11:26:06.693 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-04-25T11:26:06.716 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2024-04-25T11:26:06.733 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2024-04-25T11:26:06.734 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2024-04-25T11:26:06.790 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2024-04-25T11:26:06.791 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="9" 2024-04-25T11:26:06.791 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2024-04-25T11:26:06.791 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2024-04-25T11:26:06.791 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="9" 2024-04-25T11:26:06.791 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el9" 2024-04-25T11:26:06.791 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T11:26:06.791 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2024-04-25T11:26:06.791 INFO:teuthology.orchestra.run.smithi111.stdout:LOGO="fedora-logo-icon" 2024-04-25T11:26:06.791 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T11:26:06.791 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2024-04-25T11:26:06.791 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T11:26:06.791 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T11:26:06.791 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T11:26:06.792 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2024-04-25T11:26:06.816 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T11:26:06.830 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T11:26:06.841 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T11:26:06.841 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-04-25T11:26:06.843 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-04-25T11:26:06.845 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2024-04-25T11:26:06.861 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T11:26:06.872 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T11:26:06.872 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-04-25T11:26:06.901 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-04-25T11:26:06.903 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2024-04-25T11:26:07.014 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T11:26:07.091 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T11:26:07.106 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T11:26:07.107 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T11:26:07.107 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T11:26:07.107 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T11:26:07.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:26:07.124 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-04-25T11:26:07.180 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-437.el9.x86_64 2024-04-25T11:26:07.181 INFO:teuthology.task.kernel:Running kernel on smithi083: 5.14.0-437.el9.x86_64 2024-04-25T11:26:07.181 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum install -y kernel 2024-04-25T11:26:09.810 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 8.9 MB/s | 8.0 MB 00:00 2024-04-25T11:26:12.742 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-04-25T11:26:19.242 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.2 MB 00:02 2024-04-25T11:26:20.910 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-04-25T11:26:22.203 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-25T11:26:27.220 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-25T11:26:29.109 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T11:26:29.109 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T11:26:29.197 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-25T11:26:29.198 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-04-25T11:26:29.198 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-04-25T11:26:29.302 DEBUG:teuthology.orchestra.run.smithi083:> echo no | sudo yum reinstall kernel || true 2024-04-25T11:26:29.801 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 11:26:27 AM UTC. 2024-04-25T11:26:29.927 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-25T11:26:29.938 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-04-25T11:26:29.938 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2024-04-25T11:26:29.939 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-04-25T11:26:29.939 INFO:teuthology.orchestra.run.smithi083.stdout:Reinstalling: 2024-04-25T11:26:29.939 INFO:teuthology.orchestra.run.smithi083.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T11:26:29.939 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:26:29.939 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-04-25T11:26:29.939 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-04-25T11:26:29.939 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:26:29.939 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 255 k 2024-04-25T11:26:29.939 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 0 2024-04-25T11:26:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:Operation aborted. 2024-04-25T11:26:29.989 INFO:teuthology.orchestra.run.smithi083.stdout:Is this ok [y/N]: 2024-04-25T11:26:29.990 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum reinstall -y kernel || true 2024-04-25T11:26:30.479 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 11:26:27 AM UTC. 2024-04-25T11:26:30.595 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-25T11:26:30.595 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-04-25T11:26:30.595 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2024-04-25T11:26:30.595 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-04-25T11:26:30.595 INFO:teuthology.orchestra.run.smithi083.stdout:Reinstalling: 2024-04-25T11:26:30.595 INFO:teuthology.orchestra.run.smithi083.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T11:26:30.595 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:26:30.595 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-04-25T11:26:30.596 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-04-25T11:26:30.596 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:26:30.596 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 255 k 2024-04-25T11:26:30.596 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 0 2024-04-25T11:26:30.596 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-04-25T11:26:31.113 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.1 MB/s | 255 kB 00:00 2024-04-25T11:26:31.114 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:26:31.114 INFO:teuthology.orchestra.run.smithi083.stdout:Total 492 kB/s | 255 kB 00:00 2024-04-25T11:26:31.136 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-04-25T11:26:31.319 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-04-25T11:26:31.319 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-04-25T11:26:31.324 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-04-25T11:26:31.324 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-04-25T11:26:31.454 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-04-25T11:26:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T11:26:31.539 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:26:32.172 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:26:32.173 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T11:26:32.866 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T11:26:32.866 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:26:32.866 INFO:teuthology.orchestra.run.smithi083.stdout:Reinstalled: 2024-04-25T11:26:32.866 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T11:26:32.866 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:26:32.866 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-04-25T11:26:33.037 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T11:26:33.084 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T11:26:33.084 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T11:26:33.084 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T11:26:33.084 DEBUG:teuthology.parallel:result is None 2024-04-25T11:26:33.084 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T11:26:33.093 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T11:26:33.094 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:26:33.096 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:26:33.098 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:26:33.118 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T11:26:33.129 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T11:26:33.139 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T11:26:33.139 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:26:33.151 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:26:33.156 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:26:33.274 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T11:26:33.282 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T11:26:33.282 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T11:26:33.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:26:33.298 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T11:26:33.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:26:33.315 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:26:33.317 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:26:33.341 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:26:33.356 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:26:33.387 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:26:33.392 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:26:33.394 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:26:33.411 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:26:33.433 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:26:33.435 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T11:26:33.443 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T11:26:33.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:26:33.446 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:26:33.448 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:26:33.501 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T11:26:33.521 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T11:26:33.521 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:26:33.524 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:26:33.526 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:26:33.557 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:26:33.616 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:26:33.667 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:26:33.747 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:26:33.786 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:26:33.786 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:26:33.848 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:26:33.893 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:26:33.941 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:26:34.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:26:34.066 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T11:26:34.066 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:26:34.131 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:26:34.175 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:26:34.224 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:26:34.308 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:26:34.349 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-25T11:26:34.349 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:26:34.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-04-25T11:26:34.417 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-04-25T11:26:34.419 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2024-04-25T11:26:34.446 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:26:34.452 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:26:34.494 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:26:34.947 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T11:26:34.955 INFO:teuthology.task.internal:Starting timer... 2024-04-25T11:26:34.955 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T11:26:35.058 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T11:26:35.066 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T11:26:35.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2024-04-25T11:26:35.119 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2024-04-25T11:26:35.121 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-04-25T11:26:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-04-25T11:26:35.171 DEBUG:teuthology.orchestra.run.smithi111:> sudo service auditd rotate 2024-04-25T11:26:35.222 INFO:teuthology.orchestra.run.smithi111.stdout:Rotating logs: 2024-04-25T11:26:35.224 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T11:26:35.224 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2024-04-25T11:26:35.243 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2024-04-25T11:26:35.243 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-04-25T11:26:35.316 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-04-25T11:26:35.317 DEBUG:teuthology.orchestra.run.smithi111:> /usr/sbin/getenforce 2024-04-25T11:26:35.341 INFO:teuthology.orchestra.run.smithi111.stdout:Permissive 2024-04-25T11:26:35.341 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi083.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi094.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi111.front.sepia.ceph.com': 'permissive'} 2024-04-25T11:26:35.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:26:35.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:26:35.368 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:26:35.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:26:35.396 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:26:35.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:26:35.423 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T11:26:35.423 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2024-04-25T11:26:35.448 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-04-25T11:26:35.494 DEBUG:teuthology.orchestra.run.smithi111:> sudo /usr/sbin/setenforce permissive 2024-04-25T11:26:35.523 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T11:26:35.532 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T11:26:35.548 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T11:26:35.549 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi083.front.sepia.ceph.com,smithi094.front.sepia.ceph.com,smithi111.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T11:30:21.301 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi111.front.sepia.ceph.com')] 2024-04-25T11:30:21.302 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-25T11:30:21.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:30:21.378 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-25T11:30:21.457 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-25T11:30:21.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T11:30:21.458 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:30:21.529 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-04-25T11:30:21.604 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T11:30:21.605 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-25T11:30:21.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:30:21.678 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-04-25T11:30:21.761 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-25T11:30:21.761 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T11:30:21.773 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T11:30:21.773 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:30:21.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:30:21.776 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:30:21.776 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:30:21.778 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:30:21.778 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:30:21.802 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T11:30:21.804 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T11:30:21.817 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T11:30:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T11:30:21.829 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T11:30:21.839 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-04-25T11:30:21.840 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2024-04-25T11:30:21.842 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T11:30:21.849 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-04-25T11:30:21.850 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2024-04-25T11:30:21.862 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T11:30:21.863 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T11:30:21.865 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: ntpd: command not found 2024-04-25T11:30:21.875 INFO:teuthology.orchestra.run.smithi111.stdout:506 Cannot talk to daemon 2024-04-25T11:30:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T11:30:21.876 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T11:30:21.888 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T11:30:21.900 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T11:30:21.927 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:30:21.930 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:30:21.930 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-04-25T11:30:21.931 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:30:21.931 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:30:21.931 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:30:21.931 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:30:21.933 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:30:21.937 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:30:21.937 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-04-25T11:30:21.937 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:30:21.937 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:30:21.937 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:30:21.937 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:30:21.964 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:30:21.968 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:30:21.968 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-04-25T11:30:21.968 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:30:21.968 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:30:21.968 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:30:21.968 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:30:21.968 INFO:teuthology.run_tasks:Running task install... 2024-04-25T11:30:21.979 DEBUG:teuthology.task.install:project ceph 2024-04-25T11:30:21.979 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T11:30:21.979 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T11:30:21.979 INFO:teuthology.task.install:Using flavor: default 2024-04-25T11:30:21.985 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T11:30:21.985 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T11:30:21.986 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T11:30:21.986 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:30:21.988 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T11:30:21.988 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:30:21.989 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T11:30:21.989 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:30:22.133 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T11:30:22.133 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T11:30:22.142 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T11:30:22.142 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T11:30:22.228 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T11:30:22.228 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T11:30:22.255 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:30:22.256 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-25T11:30:22.256 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:30:22.285 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T11:30:22.285 DEBUG:teuthology.orchestra.run.smithi111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:30:22.349 DEBUG:teuthology.orchestra.run.smithi111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:30:22.375 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:30:22.375 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T11:30:22.375 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:30:22.401 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T11:30:22.401 DEBUG:teuthology.orchestra.run.smithi094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:30:22.422 DEBUG:teuthology.orchestra.run.smithi111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:30:22.425 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:30:22.425 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:30:22.425 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:30:22.450 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T11:30:22.451 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:30:22.466 DEBUG:teuthology.orchestra.run.smithi094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:30:22.484 INFO:teuthology.orchestra.run.smithi111.stdout:check_obsoletes = 1 2024-04-25T11:30:22.485 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum clean all 2024-04-25T11:30:22.515 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:30:22.539 DEBUG:teuthology.orchestra.run.smithi094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:30:22.588 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:30:22.602 INFO:teuthology.orchestra.run.smithi094.stdout:check_obsoletes = 1 2024-04-25T11:30:22.604 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum clean all 2024-04-25T11:30:22.651 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2024-04-25T11:30:22.653 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2024-04-25T11:30:22.709 INFO:teuthology.orchestra.run.smithi111.stdout:48 files removed 2024-04-25T11:30:22.738 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T11:30:22.828 INFO:teuthology.orchestra.run.smithi094.stdout:48 files removed 2024-04-25T11:30:22.855 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T11:30:22.877 INFO:teuthology.orchestra.run.smithi083.stdout:48 files removed 2024-04-25T11:30:22.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T11:30:23.397 INFO:teuthology.orchestra.run.smithi111.stdout:ceph packages for x86_64 208 kB/s | 83 kB 00:00 2024-04-25T11:30:23.466 INFO:teuthology.orchestra.run.smithi094.stdout:ceph packages for x86_64 237 kB/s | 83 kB 00:00 2024-04-25T11:30:23.634 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 178 kB/s | 83 kB 00:00 2024-04-25T11:30:23.756 INFO:teuthology.orchestra.run.smithi111.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-25T11:30:23.809 INFO:teuthology.orchestra.run.smithi094.stdout:ceph noarch packages 39 kB/s | 12 kB 00:00 2024-04-25T11:30:23.993 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-25T11:30:24.107 INFO:teuthology.orchestra.run.smithi111.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-25T11:30:24.218 INFO:teuthology.orchestra.run.smithi094.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-04-25T11:30:24.360 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-25T11:30:24.968 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T11:30:25.065 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 8.7 MB/s | 8.0 MB 00:00 2024-04-25T11:30:25.494 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 7.3 MB/s | 8.0 MB 00:01 2024-04-25T11:30:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-25T11:30:28.253 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-25T11:30:31.544 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 3.5 MB/s | 19 MB 00:05 2024-04-25T11:30:36.099 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 1.3 MB/s | 6.2 MB 00:04 2024-04-25T11:30:36.178 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 6.2 MB 00:04 2024-04-25T11:30:36.220 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 6.2 MB 00:01 2024-04-25T11:30:37.638 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 16 kB 00:00 2024-04-25T11:30:37.720 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-04-25T11:30:38.502 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-25T11:30:39.746 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-25T11:30:44.718 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-25T11:30:45.255 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 2.8 MB/s | 21 MB 00:07 2024-04-25T11:30:46.827 INFO:teuthology.orchestra.run.smithi111.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:30:46.828 INFO:teuthology.orchestra.run.smithi111.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:30:46.838 INFO:teuthology.orchestra.run.smithi111.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T11:30:46.839 INFO:teuthology.orchestra.run.smithi111.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T11:30:46.840 INFO:teuthology.orchestra.run.smithi111.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T11:30:46.841 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T11:30:46.842 INFO:teuthology.orchestra.run.smithi111.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T11:30:46.844 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T11:30:46.845 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T11:30:46.904 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-04-25T11:30:46.925 INFO:teuthology.orchestra.run.smithi111.stdout:======================================================================================= 2024-04-25T11:30:46.925 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-04-25T11:30:46.925 INFO:teuthology.orchestra.run.smithi111.stdout:======================================================================================= 2024-04-25T11:30:46.925 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-04-25T11:30:46.925 INFO:teuthology.orchestra.run.smithi111.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T11:30:46.926 INFO:teuthology.orchestra.run.smithi111.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T11:30:46.926 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:30:46.926 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T11:30:46.926 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T11:30:46.926 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T11:30:46.926 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T11:30:46.926 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T11:30:46.926 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T11:30:46.926 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T11:30:46.926 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T11:30:46.926 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T11:30:46.926 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T11:30:46.927 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T11:30:46.927 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T11:30:46.927 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T11:30:46.927 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T11:30:46.927 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T11:30:46.927 INFO:teuthology.orchestra.run.smithi111.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T11:30:46.927 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T11:30:46.927 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T11:30:46.927 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T11:30:46.927 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T11:30:46.927 INFO:teuthology.orchestra.run.smithi111.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T11:30:46.927 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T11:30:46.927 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T11:30:46.928 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T11:30:46.928 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T11:30:46.928 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T11:30:46.928 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T11:30:46.928 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T11:30:46.928 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T11:30:46.928 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T11:30:46.928 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T11:30:46.928 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:30:46.928 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T11:30:46.928 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T11:30:46.928 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2024-04-25T11:30:46.929 INFO:teuthology.orchestra.run.smithi111.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T11:30:46.929 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:30:46.929 INFO:teuthology.orchestra.run.smithi111.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T11:30:46.929 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T11:30:46.929 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-04-25T11:30:46.929 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T11:30:46.929 INFO:teuthology.orchestra.run.smithi111.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T11:30:46.929 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T11:30:46.929 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T11:30:46.929 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T11:30:46.929 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T11:30:46.929 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T11:30:46.929 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T11:30:46.930 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T11:30:46.930 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T11:30:46.930 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T11:30:46.930 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T11:30:46.930 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T11:30:46.930 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T11:30:46.930 INFO:teuthology.orchestra.run.smithi111.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T11:30:46.930 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T11:30:46.930 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T11:30:46.930 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T11:30:46.930 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T11:30:46.930 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T11:30:46.930 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T11:30:46.931 INFO:teuthology.orchestra.run.smithi111.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T11:30:46.931 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T11:30:46.931 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T11:30:46.931 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T11:30:46.931 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T11:30:46.931 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T11:30:46.931 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:30:46.931 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T11:30:46.931 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T11:30:46.931 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T11:30:46.931 INFO:teuthology.orchestra.run.smithi111.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T11:30:46.931 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T11:30:46.931 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T11:30:46.932 INFO:teuthology.orchestra.run.smithi111.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T11:30:46.932 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T11:30:46.932 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T11:30:46.932 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T11:30:46.932 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T11:30:46.932 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T11:30:46.932 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T11:30:46.932 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T11:30:46.932 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T11:30:46.932 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T11:30:46.932 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T11:30:46.932 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T11:30:46.932 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T11:30:46.932 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T11:30:46.933 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T11:30:46.933 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T11:30:46.933 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T11:30:46.933 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T11:30:46.933 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T11:30:46.933 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T11:30:46.933 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T11:30:46.933 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T11:30:46.933 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T11:30:46.933 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T11:30:46.933 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T11:30:46.933 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T11:30:46.933 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T11:30:46.934 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T11:30:46.934 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T11:30:46.934 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T11:30:46.934 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T11:30:46.934 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T11:30:46.934 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T11:30:46.934 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T11:30:46.934 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T11:30:46.934 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T11:30:46.934 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T11:30:46.934 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T11:30:46.934 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T11:30:46.934 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T11:30:46.934 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T11:30:46.935 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T11:30:46.935 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T11:30:46.935 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T11:30:46.935 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T11:30:46.935 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T11:30:46.935 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T11:30:46.935 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T11:30:46.935 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T11:30:46.935 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T11:30:46.935 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T11:30:46.935 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T11:30:46.935 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T11:30:46.935 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T11:30:46.935 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T11:30:46.935 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T11:30:46.936 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T11:30:46.936 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T11:30:46.936 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T11:30:46.936 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T11:30:46.936 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T11:30:46.936 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T11:30:46.936 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T11:30:46.936 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T11:30:46.936 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T11:30:46.936 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T11:30:46.936 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T11:30:46.936 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T11:30:46.936 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T11:30:46.936 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T11:30:46.936 INFO:teuthology.orchestra.run.smithi111.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T11:30:46.937 INFO:teuthology.orchestra.run.smithi111.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T11:30:46.937 INFO:teuthology.orchestra.run.smithi111.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T11:30:46.937 INFO:teuthology.orchestra.run.smithi111.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T11:30:46.937 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T11:30:46.937 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T11:30:46.937 INFO:teuthology.orchestra.run.smithi111.stdout:Installing weak dependencies: 2024-04-25T11:30:46.937 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T11:30:46.937 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:30:46.937 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-04-25T11:30:46.937 INFO:teuthology.orchestra.run.smithi111.stdout:======================================================================================= 2024-04-25T11:30:46.937 INFO:teuthology.orchestra.run.smithi111.stdout:Install 148 Packages 2024-04-25T11:30:46.937 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 4 Packages 2024-04-25T11:30:46.937 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:30:46.938 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 222 M 2024-04-25T11:30:46.938 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-04-25T11:30:48.278 INFO:teuthology.orchestra.run.smithi111.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-25T11:30:48.861 INFO:teuthology.orchestra.run.smithi111.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.5 MB/s | 890 kB 00:00 2024-04-25T11:30:48.961 INFO:teuthology.orchestra.run.smithi111.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.4 MB/s | 145 kB 00:00 2024-04-25T11:30:49.578 INFO:teuthology.orchestra.run.smithi111.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 3.8 MB/s | 2.4 MB 00:00 2024-04-25T11:30:49.987 INFO:teuthology.orchestra.run.smithi111.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.7 MB/s | 1.5 MB 00:00 2024-04-25T11:30:50.087 INFO:teuthology.orchestra.run.smithi111.stdout:(6/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.5 MB/s | 5.4 MB 00:02 2024-04-25T11:30:50.238 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-25T11:30:50.829 INFO:teuthology.orchestra.run.smithi111.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 5.6 MB/s | 4.7 MB 00:00 2024-04-25T11:30:51.038 INFO:teuthology.orchestra.run.smithi111.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 6.7 MB/s | 21 MB 00:03 2024-04-25T11:30:51.105 INFO:teuthology.orchestra.run.smithi111.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 379 kB/s | 25 kB 00:00 2024-04-25T11:30:51.771 INFO:teuthology.orchestra.run.smithi111.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 11 MB/s | 11 MB 00:00 2024-04-25T11:30:51.846 INFO:teuthology.orchestra.run.smithi111.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.9 MB/s | 225 kB 00:00 2024-04-25T11:30:51.922 INFO:teuthology.orchestra.run.smithi111.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 362 kB/s | 27 kB 00:00 2024-04-25T11:30:51.997 INFO:teuthology.orchestra.run.smithi111.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 423 kB/s | 32 kB 00:00 2024-04-25T11:30:52.114 INFO:teuthology.orchestra.run.smithi111.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 6.3 MB/s | 747 kB 00:00 2024-04-25T11:30:52.197 INFO:teuthology.orchestra.run.smithi111.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.9 MB/s | 163 kB 00:00 2024-04-25T11:30:52.281 INFO:teuthology.orchestra.run.smithi111.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.5 MB/s | 126 kB 00:00 2024-04-25T11:30:52.373 INFO:teuthology.orchestra.run.smithi111.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 5.3 MB/s | 500 kB 00:00 2024-04-25T11:30:52.380 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:30:52.381 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:30:52.392 INFO:teuthology.orchestra.run.smithi083.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T11:30:52.392 INFO:teuthology.orchestra.run.smithi083.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T11:30:52.393 INFO:teuthology.orchestra.run.smithi083.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T11:30:52.394 INFO:teuthology.orchestra.run.smithi083.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T11:30:52.396 INFO:teuthology.orchestra.run.smithi083.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T11:30:52.398 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T11:30:52.399 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T11:30:52.457 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-25T11:30:52.466 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-25T11:30:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-04-25T11:30:52.467 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-25T11:30:52.467 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-04-25T11:30:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T11:30:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T11:30:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:30:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T11:30:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T11:30:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T11:30:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T11:30:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T11:30:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T11:30:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T11:30:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T11:30:52.467 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T11:30:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T11:30:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T11:30:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T11:30:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T11:30:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T11:30:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T11:30:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T11:30:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T11:30:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T11:30:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T11:30:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T11:30:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T11:30:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T11:30:52.468 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T11:30:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T11:30:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T11:30:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T11:30:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T11:30:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T11:30:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T11:30:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T11:30:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T11:30:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:30:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T11:30:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T11:30:52.469 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2024-04-25T11:30:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T11:30:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:30:52.469 INFO:teuthology.orchestra.run.smithi083.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T11:30:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T11:30:52.470 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-04-25T11:30:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T11:30:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T11:30:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T11:30:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T11:30:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T11:30:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T11:30:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T11:30:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T11:30:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T11:30:52.470 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T11:30:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T11:30:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T11:30:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T11:30:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T11:30:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T11:30:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T11:30:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T11:30:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T11:30:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T11:30:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T11:30:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T11:30:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T11:30:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T11:30:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T11:30:52.471 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T11:30:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T11:30:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T11:30:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:30:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T11:30:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T11:30:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T11:30:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T11:30:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T11:30:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T11:30:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T11:30:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T11:30:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T11:30:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T11:30:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T11:30:52.472 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T11:30:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T11:30:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T11:30:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T11:30:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T11:30:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T11:30:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T11:30:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T11:30:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T11:30:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T11:30:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T11:30:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T11:30:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T11:30:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T11:30:52.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T11:30:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T11:30:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T11:30:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T11:30:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T11:30:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T11:30:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T11:30:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T11:30:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T11:30:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T11:30:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T11:30:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T11:30:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T11:30:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T11:30:52.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T11:30:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T11:30:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T11:30:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T11:30:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T11:30:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T11:30:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T11:30:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T11:30:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T11:30:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T11:30:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T11:30:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T11:30:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T11:30:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T11:30:52.475 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T11:30:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T11:30:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T11:30:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T11:30:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T11:30:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T11:30:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T11:30:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T11:30:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T11:30:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T11:30:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T11:30:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T11:30:52.476 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T11:30:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T11:30:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T11:30:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T11:30:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T11:30:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T11:30:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T11:30:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T11:30:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T11:30:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T11:30:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T11:30:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T11:30:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T11:30:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T11:30:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T11:30:52.477 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T11:30:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T11:30:52.478 INFO:teuthology.orchestra.run.smithi083.stdout:Installing weak dependencies: 2024-04-25T11:30:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T11:30:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:30:52.478 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-04-25T11:30:52.478 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-25T11:30:52.478 INFO:teuthology.orchestra.run.smithi083.stdout:Install 148 Packages 2024-04-25T11:30:52.478 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 4 Packages 2024-04-25T11:30:52.478 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:30:52.479 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 222 M 2024-04-25T11:30:52.479 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-04-25T11:30:53.157 INFO:teuthology.orchestra.run.smithi111.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 6.8 MB/s | 5.4 MB 00:00 2024-04-25T11:30:53.240 INFO:teuthology.orchestra.run.smithi111.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 543 kB/s | 45 kB 00:00 2024-04-25T11:30:53.407 INFO:teuthology.orchestra.run.smithi111.stdout:(20/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 5.1 MB/s | 17 MB 00:03 2024-04-25T11:30:53.433 INFO:teuthology.orchestra.run.smithi111.stdout:(21/152): python3-ceph-common-19.0.0-3156.gd69f 701 kB/s | 135 kB 00:00 2024-04-25T11:30:53.500 INFO:teuthology.orchestra.run.smithi111.stdout:(22/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.7 MB/s | 161 kB 00:00 2024-04-25T11:30:53.631 INFO:teuthology.orchestra.run.smithi083.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-25T11:30:53.650 INFO:teuthology.orchestra.run.smithi111.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.9 MB/s | 298 kB 00:00 2024-04-25T11:30:53.675 INFO:teuthology.orchestra.run.smithi111.stdout:(24/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.3 MB/s | 320 kB 00:00 2024-04-25T11:30:53.759 INFO:teuthology.orchestra.run.smithi111.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 915 kB/s | 99 kB 00:00 2024-04-25T11:30:53.784 INFO:teuthology.orchestra.run.smithi111.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 786 kB/s | 85 kB 00:00 2024-04-25T11:30:53.884 INFO:teuthology.orchestra.run.smithi111.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.7 MB/s | 170 kB 00:00 2024-04-25T11:30:53.985 INFO:teuthology.orchestra.run.smithi111.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 342 kB/s | 34 kB 00:00 2024-04-25T11:30:54.085 INFO:teuthology.orchestra.run.smithi111.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.4 MB/s | 142 kB 00:00 2024-04-25T11:30:54.306 INFO:teuthology.orchestra.run.smithi083.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.3 MB/s | 890 kB 00:00 2024-04-25T11:30:54.398 INFO:teuthology.orchestra.run.smithi083.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.5 MB/s | 145 kB 00:00 2024-04-25T11:30:54.827 INFO:teuthology.orchestra.run.smithi111.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 4.8 MB/s | 3.5 MB 00:00 2024-04-25T11:30:54.960 INFO:teuthology.orchestra.run.smithi111.stdout:(31/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 2.6 MB/s | 3.1 MB 00:01 2024-04-25T11:30:55.161 INFO:teuthology.orchestra.run.smithi111.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.2 MB/s | 249 kB 00:00 2024-04-25T11:30:55.232 INFO:teuthology.orchestra.run.smithi083.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 2.8 MB/s | 2.4 MB 00:00 2024-04-25T11:30:55.261 INFO:teuthology.orchestra.run.smithi111.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 493 kB/s | 49 kB 00:00 2024-04-25T11:30:55.428 INFO:teuthology.orchestra.run.smithi111.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 99 kB/s | 16 kB 00:00 2024-04-25T11:30:55.629 INFO:teuthology.orchestra.run.smithi111.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 273 kB 00:00 2024-04-25T11:30:55.690 INFO:teuthology.orchestra.run.smithi083.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.3 MB/s | 1.5 MB 00:00 2024-04-25T11:30:55.962 INFO:teuthology.orchestra.run.smithi111.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 2.2 MB/s | 762 kB 00:00 2024-04-25T11:30:56.063 INFO:teuthology.orchestra.run.smithi111.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 182 kB/s | 18 kB 00:00 2024-04-25T11:30:56.246 INFO:teuthology.orchestra.run.smithi111.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 438 kB/s | 80 kB 00:00 2024-04-25T11:30:56.430 INFO:teuthology.orchestra.run.smithi111.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-04-25T11:30:56.488 INFO:teuthology.orchestra.run.smithi111.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-04-25T11:30:56.572 INFO:teuthology.orchestra.run.smithi111.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-04-25T11:30:56.780 INFO:teuthology.orchestra.run.smithi111.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 5.9 MB/s | 1.2 MB 00:00 2024-04-25T11:30:56.839 INFO:teuthology.orchestra.run.smithi111.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T11:30:56.931 INFO:teuthology.orchestra.run.smithi111.stdout:(44/152): ceph-mgr-diskprediction-local-19.0.0- 3.5 MB/s | 7.4 MB 00:02 2024-04-25T11:30:56.941 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-25T11:30:56.956 INFO:teuthology.orchestra.run.smithi111.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-04-25T11:30:57.015 INFO:teuthology.orchestra.run.smithi111.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T11:30:57.099 INFO:teuthology.orchestra.run.smithi083.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 3.3 MB/s | 4.7 MB 00:01 2024-04-25T11:30:57.115 INFO:teuthology.orchestra.run.smithi111.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 679 kB/s | 125 kB 00:00 2024-04-25T11:30:57.207 INFO:teuthology.orchestra.run.smithi111.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-04-25T11:30:57.266 INFO:teuthology.orchestra.run.smithi111.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T11:30:57.308 INFO:teuthology.orchestra.run.smithi111.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 517 kB/s | 21 kB 00:00 2024-04-25T11:30:57.358 INFO:teuthology.orchestra.run.smithi111.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T11:30:57.392 INFO:teuthology.orchestra.run.smithi111.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-04-25T11:30:57.467 INFO:teuthology.orchestra.run.smithi111.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-04-25T11:30:57.509 INFO:teuthology.orchestra.run.smithi111.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 436 kB/s | 18 kB 00:00 2024-04-25T11:30:57.541 INFO:teuthology.orchestra.run.smithi083.stdout:(7/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.3 MB/s | 5.4 MB 00:04 2024-04-25T11:30:57.560 INFO:teuthology.orchestra.run.smithi111.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-04-25T11:30:57.585 INFO:teuthology.orchestra.run.smithi111.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-04-25T11:30:57.610 INFO:teuthology.orchestra.run.smithi111.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T11:30:57.635 INFO:teuthology.orchestra.run.smithi111.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-04-25T11:30:57.669 INFO:teuthology.orchestra.run.smithi111.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T11:30:57.694 INFO:teuthology.orchestra.run.smithi111.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-04-25T11:30:57.719 INFO:teuthology.orchestra.run.smithi111.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 508 kB/s | 25 kB 00:00 2024-04-25T11:30:57.745 INFO:teuthology.orchestra.run.smithi111.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-04-25T11:30:57.770 INFO:teuthology.orchestra.run.smithi111.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T11:30:57.795 INFO:teuthology.orchestra.run.smithi111.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T11:30:57.821 INFO:teuthology.orchestra.run.smithi111.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-04-25T11:30:57.946 INFO:teuthology.orchestra.run.smithi111.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-04-25T11:30:58.021 INFO:teuthology.orchestra.run.smithi111.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-25T11:30:58.088 INFO:teuthology.orchestra.run.smithi111.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-25T11:30:58.122 INFO:teuthology.orchestra.run.smithi111.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-04-25T11:30:58.139 INFO:teuthology.orchestra.run.smithi111.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-04-25T11:30:58.306 INFO:teuthology.orchestra.run.smithi111.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 32 MB/s | 5.8 MB 00:00 2024-04-25T11:30:58.356 INFO:teuthology.orchestra.run.smithi111.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-25T11:30:58.531 INFO:teuthology.orchestra.run.smithi111.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-04-25T11:30:58.565 INFO:teuthology.orchestra.run.smithi111.stdout:(74/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.4 MB/s | 1.0 MB 00:00 2024-04-25T11:30:58.590 INFO:teuthology.orchestra.run.smithi111.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-25T11:30:58.616 INFO:teuthology.orchestra.run.smithi111.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-04-25T11:30:58.633 INFO:teuthology.orchestra.run.smithi111.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 610 kB/s | 26 kB 00:00 2024-04-25T11:30:58.658 INFO:teuthology.orchestra.run.smithi111.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 455 kB/s | 19 kB 00:00 2024-04-25T11:30:58.683 INFO:teuthology.orchestra.run.smithi111.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-25T11:30:58.700 INFO:teuthology.orchestra.run.smithi111.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 829 kB/s | 35 kB 00:00 2024-04-25T11:30:58.767 INFO:teuthology.orchestra.run.smithi111.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-25T11:30:58.834 INFO:teuthology.orchestra.run.smithi111.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-04-25T11:30:58.859 INFO:teuthology.orchestra.run.smithi111.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-25T11:30:58.884 INFO:teuthology.orchestra.run.smithi111.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T11:30:58.901 INFO:teuthology.orchestra.run.smithi111.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-25T11:30:58.935 INFO:teuthology.orchestra.run.smithi111.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T11:30:58.960 INFO:teuthology.orchestra.run.smithi111.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-04-25T11:30:58.986 INFO:teuthology.orchestra.run.smithi111.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T11:30:59.011 INFO:teuthology.orchestra.run.smithi111.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T11:30:59.036 INFO:teuthology.orchestra.run.smithi111.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T11:30:59.086 INFO:teuthology.orchestra.run.smithi111.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 834 kB/s | 42 kB 00:00 2024-04-25T11:30:59.137 INFO:teuthology.orchestra.run.smithi111.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-04-25T11:30:59.196 INFO:teuthology.orchestra.run.smithi111.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-25T11:30:59.246 INFO:teuthology.orchestra.run.smithi111.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-04-25T11:30:59.296 INFO:teuthology.orchestra.run.smithi111.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-25T11:30:59.355 INFO:teuthology.orchestra.run.smithi111.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T11:30:59.405 INFO:teuthology.orchestra.run.smithi111.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T11:30:59.464 INFO:teuthology.orchestra.run.smithi111.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T11:30:59.514 INFO:teuthology.orchestra.run.smithi111.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T11:30:59.706 INFO:teuthology.orchestra.run.smithi111.stdout:(100/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-25T11:30:59.731 INFO:teuthology.orchestra.run.smithi111.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 207 kB/s | 45 kB 00:00 2024-04-25T11:30:59.790 INFO:teuthology.orchestra.run.smithi111.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-04-25T11:31:00.065 INFO:teuthology.orchestra.run.smithi111.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-04-25T11:31:00.410 INFO:teuthology.orchestra.run.smithi111.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 889 kB/s | 548 kB 00:00 2024-04-25T11:31:00.524 INFO:teuthology.orchestra.run.smithi111.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 166 kB/s | 19 kB 00:00 2024-04-25T11:31:00.583 INFO:teuthology.orchestra.run.smithi111.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 597 kB/s | 309 kB 00:00 2024-04-25T11:31:00.683 INFO:teuthology.orchestra.run.smithi111.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-04-25T11:31:00.724 INFO:teuthology.orchestra.run.smithi083.stdout:(8/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 4.6 MB/s | 17 MB 00:03 2024-04-25T11:31:00.783 INFO:teuthology.orchestra.run.smithi111.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 487 kB/s | 49 kB 00:00 2024-04-25T11:31:00.816 INFO:teuthology.orchestra.run.smithi083.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 276 kB/s | 25 kB 00:00 2024-04-25T11:31:00.934 INFO:teuthology.orchestra.run.smithi111.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-25T11:31:00.959 INFO:teuthology.orchestra.run.smithi111.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 384 kB/s | 67 kB 00:00 2024-04-25T11:31:01.042 INFO:teuthology.orchestra.run.smithi111.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-25T11:31:01.176 INFO:teuthology.orchestra.run.smithi111.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-04-25T11:31:01.210 INFO:teuthology.orchestra.run.smithi111.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 838 kB 00:00 2024-04-25T11:31:01.276 INFO:teuthology.orchestra.run.smithi111.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 435 kB/s | 43 kB 00:00 2024-04-25T11:31:01.302 INFO:teuthology.orchestra.run.smithi111.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-04-25T11:31:01.369 INFO:teuthology.orchestra.run.smithi111.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-25T11:31:01.411 INFO:teuthology.orchestra.run.smithi111.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-25T11:31:01.469 INFO:teuthology.orchestra.run.smithi111.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-25T11:31:01.528 INFO:teuthology.orchestra.run.smithi111.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-25T11:31:01.561 INFO:teuthology.orchestra.run.smithi111.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-04-25T11:31:01.637 INFO:teuthology.orchestra.run.smithi111.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.9 MB/s | 206 kB 00:00 2024-04-25T11:31:01.720 INFO:teuthology.orchestra.run.smithi111.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-04-25T11:31:01.745 INFO:teuthology.orchestra.run.smithi111.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-25T11:31:01.804 INFO:teuthology.orchestra.run.smithi111.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-04-25T11:31:01.829 INFO:teuthology.orchestra.run.smithi111.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-04-25T11:31:01.905 INFO:teuthology.orchestra.run.smithi111.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-25T11:31:01.921 INFO:teuthology.orchestra.run.smithi111.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-25T11:31:01.988 INFO:teuthology.orchestra.run.smithi111.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 236 kB/s | 20 kB 00:00 2024-04-25T11:31:02.005 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T11:31:02.089 INFO:teuthology.orchestra.run.smithi111.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-04-25T11:31:02.106 INFO:teuthology.orchestra.run.smithi111.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-04-25T11:31:02.189 INFO:teuthology.orchestra.run.smithi111.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 787 kB/s | 79 kB 00:00 2024-04-25T11:31:02.215 INFO:teuthology.orchestra.run.smithi111.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 532 kB/s | 58 kB 00:00 2024-04-25T11:31:02.315 INFO:teuthology.orchestra.run.smithi111.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-25T11:31:02.332 INFO:teuthology.orchestra.run.smithi111.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-04-25T11:31:02.424 INFO:teuthology.orchestra.run.smithi111.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 830 kB/s | 90 kB 00:00 2024-04-25T11:31:02.451 INFO:teuthology.orchestra.run.smithi111.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 237 kB/s | 28 kB 00:00 2024-04-25T11:31:02.618 INFO:teuthology.orchestra.run.smithi111.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 158 kB/s | 31 kB 00:00 2024-04-25T11:31:02.718 INFO:teuthology.orchestra.run.smithi111.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 706 kB/s | 188 kB 00:00 2024-04-25T11:31:02.735 INFO:teuthology.orchestra.run.smithi111.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 506 kB/s | 59 kB 00:00 2024-04-25T11:31:02.860 INFO:teuthology.orchestra.run.smithi111.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 252 kB/s | 36 kB 00:00 2024-04-25T11:31:02.877 INFO:teuthology.orchestra.run.smithi111.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 359 kB/s | 51 kB 00:00 2024-04-25T11:31:02.994 INFO:teuthology.orchestra.run.smithi111.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-04-25T11:31:03.020 INFO:teuthology.orchestra.run.smithi111.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 632 kB/s | 90 kB 00:00 2024-04-25T11:31:03.120 INFO:teuthology.orchestra.run.smithi111.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-25T11:31:03.145 INFO:teuthology.orchestra.run.smithi111.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 159 kB/s | 20 kB 00:00 2024-04-25T11:31:03.229 INFO:teuthology.orchestra.run.smithi111.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-25T11:31:03.346 INFO:teuthology.orchestra.run.smithi111.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-25T11:31:04.147 INFO:teuthology.orchestra.run.smithi094.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:31:04.148 INFO:teuthology.orchestra.run.smithi094.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:31:04.159 INFO:teuthology.orchestra.run.smithi094.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T11:31:04.160 INFO:teuthology.orchestra.run.smithi094.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T11:31:04.160 INFO:teuthology.orchestra.run.smithi094.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T11:31:04.161 INFO:teuthology.orchestra.run.smithi094.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T11:31:04.163 INFO:teuthology.orchestra.run.smithi094.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T11:31:04.165 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T11:31:04.166 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T11:31:04.226 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-25T11:31:04.243 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-25T11:31:04.243 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-04-25T11:31:04.243 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-25T11:31:04.243 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-25T11:31:04.243 INFO:teuthology.orchestra.run.smithi094.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T11:31:04.243 INFO:teuthology.orchestra.run.smithi094.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T11:31:04.244 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:31:04.244 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T11:31:04.244 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T11:31:04.244 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T11:31:04.244 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T11:31:04.244 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T11:31:04.244 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T11:31:04.244 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T11:31:04.244 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T11:31:04.244 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T11:31:04.244 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T11:31:04.244 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T11:31:04.245 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T11:31:04.245 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T11:31:04.245 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T11:31:04.245 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T11:31:04.245 INFO:teuthology.orchestra.run.smithi094.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T11:31:04.245 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T11:31:04.245 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T11:31:04.245 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T11:31:04.245 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T11:31:04.245 INFO:teuthology.orchestra.run.smithi094.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T11:31:04.245 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T11:31:04.245 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T11:31:04.245 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T11:31:04.245 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T11:31:04.246 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T11:31:04.246 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T11:31:04.246 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T11:31:04.246 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T11:31:04.246 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T11:31:04.246 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T11:31:04.246 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:31:04.246 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T11:31:04.246 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T11:31:04.246 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrading: 2024-04-25T11:31:04.246 INFO:teuthology.orchestra.run.smithi094.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T11:31:04.247 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T11:31:04.247 INFO:teuthology.orchestra.run.smithi094.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T11:31:04.247 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T11:31:04.247 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-04-25T11:31:04.247 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T11:31:04.247 INFO:teuthology.orchestra.run.smithi094.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T11:31:04.247 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T11:31:04.247 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T11:31:04.247 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T11:31:04.247 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T11:31:04.247 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T11:31:04.247 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T11:31:04.247 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi094.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T11:31:04.248 INFO:teuthology.orchestra.run.smithi094.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi094.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T11:31:04.249 INFO:teuthology.orchestra.run.smithi094.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T11:31:04.250 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T11:31:04.250 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T11:31:04.250 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T11:31:04.250 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T11:31:04.250 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T11:31:04.250 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T11:31:04.250 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T11:31:04.250 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T11:31:04.250 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T11:31:04.250 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T11:31:04.250 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T11:31:04.250 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T11:31:04.250 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T11:31:04.251 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T11:31:04.251 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T11:31:04.251 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T11:31:04.251 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T11:31:04.251 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T11:31:04.251 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T11:31:04.251 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T11:31:04.251 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T11:31:04.251 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T11:31:04.251 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T11:31:04.251 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T11:31:04.251 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T11:31:04.251 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T11:31:04.251 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T11:31:04.252 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T11:31:04.252 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T11:31:04.252 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T11:31:04.252 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T11:31:04.252 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T11:31:04.252 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T11:31:04.252 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T11:31:04.252 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T11:31:04.252 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T11:31:04.252 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T11:31:04.252 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T11:31:04.252 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T11:31:04.252 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T11:31:04.253 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T11:31:04.253 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T11:31:04.253 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T11:31:04.253 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T11:31:04.253 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T11:31:04.253 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T11:31:04.253 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T11:31:04.253 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T11:31:04.253 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T11:31:04.253 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T11:31:04.253 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T11:31:04.253 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T11:31:04.253 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T11:31:04.253 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T11:31:04.254 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T11:31:04.254 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T11:31:04.254 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T11:31:04.254 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T11:31:04.254 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T11:31:04.254 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T11:31:04.254 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T11:31:04.254 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T11:31:04.254 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T11:31:04.254 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T11:31:04.254 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T11:31:04.254 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T11:31:04.254 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T11:31:04.255 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T11:31:04.255 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T11:31:04.255 INFO:teuthology.orchestra.run.smithi094.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T11:31:04.255 INFO:teuthology.orchestra.run.smithi094.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T11:31:04.255 INFO:teuthology.orchestra.run.smithi094.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T11:31:04.255 INFO:teuthology.orchestra.run.smithi094.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T11:31:04.255 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T11:31:04.255 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T11:31:04.255 INFO:teuthology.orchestra.run.smithi094.stdout:Installing weak dependencies: 2024-04-25T11:31:04.255 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T11:31:04.255 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T11:31:04.255 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-25T11:31:04.255 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-25T11:31:04.255 INFO:teuthology.orchestra.run.smithi094.stdout:Install 148 Packages 2024-04-25T11:31:04.256 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrade 4 Packages 2024-04-25T11:31:04.256 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T11:31:04.256 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 222 M 2024-04-25T11:31:04.256 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-25T11:31:04.616 INFO:teuthology.orchestra.run.smithi083.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 1.5 MB/s | 11 MB 00:07 2024-04-25T11:31:04.916 INFO:teuthology.orchestra.run.smithi083.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 750 kB/s | 225 kB 00:00 2024-04-25T11:31:05.033 INFO:teuthology.orchestra.run.smithi083.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 232 kB/s | 27 kB 00:00 2024-04-25T11:31:05.175 INFO:teuthology.orchestra.run.smithi083.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 223 kB/s | 32 kB 00:00 2024-04-25T11:31:05.463 INFO:teuthology.orchestra.run.smithi111.stdout:(148/152): librados2-19.0.0-3156.gd69f13d8.el7. 1.5 MB/s | 3.4 MB 00:02 2024-04-25T11:31:05.590 INFO:teuthology.orchestra.run.smithi094.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 11 kB/s | 6.6 kB 00:00 2024-04-25T11:31:05.838 INFO:teuthology.orchestra.run.smithi111.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 3.6 MB/s | 1.3 MB 00:00 2024-04-25T11:31:06.163 INFO:teuthology.orchestra.run.smithi111.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 2.4 MB 00:00 2024-04-25T11:31:06.223 INFO:teuthology.orchestra.run.smithi094.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 890 kB 00:00 2024-04-25T11:31:06.250 INFO:teuthology.orchestra.run.smithi083.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 695 kB/s | 747 kB 00:01 2024-04-25T11:31:06.307 INFO:teuthology.orchestra.run.smithi094.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.7 MB/s | 145 kB 00:00 2024-04-25T11:31:06.625 INFO:teuthology.orchestra.run.smithi083.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 435 kB/s | 163 kB 00:00 2024-04-25T11:31:06.958 INFO:teuthology.orchestra.run.smithi094.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 3.6 MB/s | 2.4 MB 00:00 2024-04-25T11:31:07.053 INFO:teuthology.orchestra.run.smithi083.stdout:(16/152): ceph-common-19.0.0-3156.gd69f13d8.el7 1.5 MB/s | 21 MB 00:13 2024-04-25T11:31:07.078 INFO:teuthology.orchestra.run.smithi083.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 280 kB/s | 126 kB 00:00 2024-04-25T11:31:07.358 INFO:teuthology.orchestra.run.smithi094.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.7 MB/s | 1.5 MB 00:00 2024-04-25T11:31:07.404 INFO:teuthology.orchestra.run.smithi083.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 1.4 MB/s | 500 kB 00:00 2024-04-25T11:31:07.470 INFO:teuthology.orchestra.run.smithi083.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 678 kB/s | 45 kB 00:00 2024-04-25T11:31:07.621 INFO:teuthology.orchestra.run.smithi083.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 898 kB/s | 135 kB 00:00 2024-04-25T11:31:07.787 INFO:teuthology.orchestra.run.smithi083.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 969 kB/s | 161 kB 00:00 2024-04-25T11:31:08.005 INFO:teuthology.orchestra.run.smithi083.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.4 MB/s | 320 kB 00:00 2024-04-25T11:31:08.180 INFO:teuthology.orchestra.run.smithi083.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 298 kB 00:00 2024-04-25T11:31:08.238 INFO:teuthology.orchestra.run.smithi083.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 99 kB 00:00 2024-04-25T11:31:08.255 INFO:teuthology.orchestra.run.smithi111.stdout:(151/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 2.9 MB/s | 49 MB 00:17 2024-04-25T11:31:08.331 INFO:teuthology.orchestra.run.smithi083.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 929 kB/s | 85 kB 00:00 2024-04-25T11:31:08.392 INFO:teuthology.orchestra.run.smithi094.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 4.5 MB/s | 4.7 MB 00:01 2024-04-25T11:31:08.613 INFO:teuthology.orchestra.run.smithi111.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 611 kB/s | 3.1 MB 00:05 2024-04-25T11:31:08.617 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:31:08.617 INFO:teuthology.orchestra.run.smithi111.stdout:Total 10 MB/s | 222 MB 00:21 2024-04-25T11:31:09.677 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-04-25T11:31:09.773 INFO:teuthology.orchestra.run.smithi083.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 2.1 MB/s | 3.1 MB 00:01 2024-04-25T11:31:09.818 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-04-25T11:31:09.818 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-04-25T11:31:09.923 INFO:teuthology.orchestra.run.smithi083.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.1 MB/s | 170 kB 00:00 2024-04-25T11:31:10.007 INFO:teuthology.orchestra.run.smithi083.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 410 kB/s | 34 kB 00:00 2024-04-25T11:31:10.115 INFO:teuthology.orchestra.run.smithi083.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.3 MB/s | 142 kB 00:00 2024-04-25T11:31:11.525 INFO:teuthology.orchestra.run.smithi094.stdout:(7/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 839 kB/s | 5.4 MB 00:06 2024-04-25T11:31:11.709 INFO:teuthology.orchestra.run.smithi094.stdout:(8/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 5.1 MB/s | 17 MB 00:03 2024-04-25T11:31:11.722 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-04-25T11:31:11.722 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-04-25T11:31:11.751 INFO:teuthology.orchestra.run.smithi094.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 608 kB/s | 25 kB 00:00 2024-04-25T11:31:11.841 INFO:teuthology.orchestra.run.smithi083.stdout:(30/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 4.5 MB/s | 49 MB 00:11 2024-04-25T11:31:11.908 INFO:teuthology.orchestra.run.smithi083.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 2.0 MB/s | 3.5 MB 00:01 2024-04-25T11:31:12.058 INFO:teuthology.orchestra.run.smithi083.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.6 MB/s | 249 kB 00:00 2024-04-25T11:31:12.150 INFO:teuthology.orchestra.run.smithi083.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 538 kB/s | 49 kB 00:00 2024-04-25T11:31:12.242 INFO:teuthology.orchestra.run.smithi083.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 180 kB/s | 16 kB 00:00 2024-04-25T11:31:12.400 INFO:teuthology.orchestra.run.smithi083.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 273 kB 00:00 2024-04-25T11:31:12.717 INFO:teuthology.orchestra.run.smithi083.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 2.3 MB/s | 762 kB 00:00 2024-04-25T11:31:12.784 INFO:teuthology.orchestra.run.smithi083.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 274 kB/s | 18 kB 00:00 2024-04-25T11:31:12.959 INFO:teuthology.orchestra.run.smithi083.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 461 kB/s | 80 kB 00:00 2024-04-25T11:31:13.035 INFO:teuthology.orchestra.run.smithi083.stdout:(39/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 922 kB/s | 5.4 MB 00:05 2024-04-25T11:31:13.084 INFO:teuthology.orchestra.run.smithi094.stdout:(10/152): ceph-common-19.0.0-3156.gd69f13d8.el7 2.6 MB/s | 21 MB 00:08 2024-04-25T11:31:13.193 INFO:teuthology.orchestra.run.smithi083.stdout:(40/152): ceph-mgr-diskprediction-local-19.0.0- 5.5 MB/s | 7.4 MB 00:01 2024-04-25T11:31:13.201 INFO:teuthology.orchestra.run.smithi094.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.9 MB/s | 225 kB 00:00 2024-04-25T11:31:13.227 INFO:teuthology.orchestra.run.smithi083.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-04-25T11:31:13.260 INFO:teuthology.orchestra.run.smithi094.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 463 kB/s | 27 kB 00:00 2024-04-25T11:31:13.261 INFO:teuthology.orchestra.run.smithi083.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 866 kB/s | 195 kB 00:00 2024-04-25T11:31:13.318 INFO:teuthology.orchestra.run.smithi094.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 544 kB/s | 32 kB 00:00 2024-04-25T11:31:13.328 INFO:teuthology.orchestra.run.smithi083.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T11:31:13.395 INFO:teuthology.orchestra.run.smithi083.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-25T11:31:13.470 INFO:teuthology.orchestra.run.smithi083.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-25T11:31:13.519 INFO:teuthology.orchestra.run.smithi094.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 3.6 MB/s | 747 kB 00:00 2024-04-25T11:31:13.529 INFO:teuthology.orchestra.run.smithi083.stdout:(46/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 757 kB/s | 253 kB 00:00 2024-04-25T11:31:13.577 INFO:teuthology.orchestra.run.smithi094.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.7 MB/s | 163 kB 00:00 2024-04-25T11:31:13.604 INFO:teuthology.orchestra.run.smithi083.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-04-25T11:31:13.636 INFO:teuthology.orchestra.run.smithi094.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 2.1 MB/s | 126 kB 00:00 2024-04-25T11:31:13.646 INFO:teuthology.orchestra.run.smithi083.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 2.9 MB/s | 1.2 MB 00:00 2024-04-25T11:31:13.770 INFO:teuthology.orchestra.run.smithi094.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 3.7 MB/s | 500 kB 00:00 2024-04-25T11:31:14.096 INFO:teuthology.orchestra.run.smithi083.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 237 kB/s | 107 kB 00:00 2024-04-25T11:31:14.296 INFO:teuthology.orchestra.run.smithi083.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 898 kB/s | 689 kB 00:00 2024-04-25T11:31:14.355 INFO:teuthology.orchestra.run.smithi083.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 83 kB/s | 21 kB 00:00 2024-04-25T11:31:14.622 INFO:teuthology.orchestra.run.smithi083.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 123 kB/s | 33 kB 00:00 2024-04-25T11:31:14.805 INFO:teuthology.orchestra.run.smithi083.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 616 kB/s | 313 kB 00:00 2024-04-25T11:31:14.820 INFO:teuthology.orchestra.run.smithi094.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 5.1 MB/s | 5.4 MB 00:01 2024-04-25T11:31:14.879 INFO:teuthology.orchestra.run.smithi094.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 778 kB/s | 45 kB 00:00 2024-04-25T11:31:14.937 INFO:teuthology.orchestra.run.smithi094.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 2.3 MB/s | 135 kB 00:00 2024-04-25T11:31:14.996 INFO:teuthology.orchestra.run.smithi094.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.7 MB/s | 161 kB 00:00 2024-04-25T11:31:15.064 INFO:teuthology.orchestra.run.smithi083.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 70 kB/s | 18 kB 00:00 2024-04-25T11:31:15.088 INFO:teuthology.orchestra.run.smithi094.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.4 MB/s | 320 kB 00:00 2024-04-25T11:31:15.181 INFO:teuthology.orchestra.run.smithi094.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 298 kB 00:00 2024-04-25T11:31:15.231 INFO:teuthology.orchestra.run.smithi094.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.9 MB/s | 99 kB 00:00 2024-04-25T11:31:15.248 INFO:teuthology.orchestra.run.smithi083.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 61 kB/s | 11 kB 00:00 2024-04-25T11:31:15.290 INFO:teuthology.orchestra.run.smithi094.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 85 kB 00:00 2024-04-25T11:31:15.556 INFO:teuthology.orchestra.run.smithi083.stdout:(56/152): bison-3.7.4-5.el9.x86_64.rpm 483 kB/s | 943 kB 00:01 2024-04-25T11:31:15.656 INFO:teuthology.orchestra.run.smithi083.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 392 kB/s | 160 kB 00:00 2024-04-25T11:31:15.841 INFO:teuthology.orchestra.run.smithi094.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 5.6 MB/s | 3.1 MB 00:00 2024-04-25T11:31:15.898 INFO:teuthology.orchestra.run.smithi083.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 132 kB/s | 45 kB 00:00 2024-04-25T11:31:15.907 INFO:teuthology.orchestra.run.smithi094.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.5 MB/s | 170 kB 00:00 2024-04-25T11:31:15.949 INFO:teuthology.orchestra.run.smithi094.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 820 kB/s | 34 kB 00:00 2024-04-25T11:31:16.008 INFO:teuthology.orchestra.run.smithi094.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.4 MB/s | 142 kB 00:00 2024-04-25T11:31:16.340 INFO:teuthology.orchestra.run.smithi083.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.7 MB/s | 3.0 MB 00:01 2024-04-25T11:31:16.607 INFO:teuthology.orchestra.run.smithi083.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 822 kB/s | 582 kB 00:00 2024-04-25T11:31:16.633 INFO:teuthology.orchestra.run.smithi083.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 87 kB/s | 25 kB 00:00 2024-04-25T11:31:16.692 INFO:teuthology.orchestra.run.smithi094.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 5.2 MB/s | 3.5 MB 00:00 2024-04-25T11:31:16.949 INFO:teuthology.orchestra.run.smithi083.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 139 kB/s | 47 kB 00:00 2024-04-25T11:31:16.974 INFO:teuthology.orchestra.run.smithi083.stdout:(63/152): librdkafka-1.6.1-102.el9.x86_64.rpm 503 kB/s | 662 kB 00:01 2024-04-25T11:31:17.092 INFO:teuthology.orchestra.run.smithi094.stdout:(31/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 1.9 MB/s | 11 MB 00:05 2024-04-25T11:31:17.242 INFO:teuthology.orchestra.run.smithi083.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 480 kB/s | 292 kB 00:00 2024-04-25T11:31:17.267 INFO:teuthology.orchestra.run.smithi083.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 121 kB/s | 35 kB 00:00 2024-04-25T11:31:17.459 INFO:teuthology.orchestra.run.smithi083.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 370 kB/s | 188 kB 00:00 2024-04-25T11:31:17.568 INFO:teuthology.orchestra.run.smithi094.stdout:(32/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 8.4 MB/s | 49 MB 00:05 2024-04-25T11:31:17.593 INFO:teuthology.orchestra.run.smithi094.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3156.gd6 497 kB/s | 249 kB 00:00 2024-04-25T11:31:17.618 INFO:teuthology.orchestra.run.smithi094.stdout:(34/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 986 kB/s | 49 kB 00:00 2024-04-25T11:31:17.668 INFO:teuthology.orchestra.run.smithi094.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3156.gd 219 kB/s | 16 kB 00:00 2024-04-25T11:31:17.670 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-04-25T11:31:17.694 INFO:teuthology.orchestra.run.smithi094.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 3.5 MB/s | 273 kB 00:00 2024-04-25T11:31:17.736 INFO:teuthology.orchestra.run.smithi094.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 439 kB/s | 18 kB 00:00 2024-04-25T11:31:17.758 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T11:31:17.796 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T11:31:17.928 INFO:teuthology.orchestra.run.smithi094.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 6.0 MB/s | 7.4 MB 00:01 2024-04-25T11:31:17.952 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T11:31:17.953 INFO:teuthology.orchestra.run.smithi094.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 370 kB/s | 80 kB 00:00 2024-04-25T11:31:17.978 INFO:teuthology.orchestra.run.smithi094.stdout:(40/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 2.4 MB/s | 762 kB 00:00 2024-04-25T11:31:18.062 INFO:teuthology.orchestra.run.smithi094.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-04-25T11:31:18.111 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T11:31:18.135 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T11:31:18.180 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T11:31:18.204 INFO:teuthology.orchestra.run.smithi094.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-04-25T11:31:18.238 INFO:teuthology.orchestra.run.smithi094.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 977 kB/s | 253 kB 00:00 2024-04-25T11:31:18.288 INFO:teuthology.orchestra.run.smithi094.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-25T11:31:18.389 INFO:teuthology.orchestra.run.smithi094.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 899 kB/s | 135 kB 00:00 2024-04-25T11:31:18.414 INFO:teuthology.orchestra.run.smithi094.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 996 kB/s | 125 kB 00:00 2024-04-25T11:31:18.510 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T11:31:18.534 INFO:teuthology.orchestra.run.smithi083.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-04-25T11:31:18.547 INFO:teuthology.orchestra.run.smithi094.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.3 MB/s | 215 kB 00:00 2024-04-25T11:31:18.615 INFO:teuthology.orchestra.run.smithi094.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-04-25T11:31:18.627 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:31:18.657 INFO:teuthology.orchestra.run.smithi094.stdout:(49/152): python3-cryptography-36.0.1-4.el9.x86 2.1 MB/s | 1.2 MB 00:00 2024-04-25T11:31:18.682 INFO:teuthology.orchestra.run.smithi094.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T11:31:18.699 INFO:teuthology.orchestra.run.smithi094.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 514 kB/s | 21 kB 00:00 2024-04-25T11:31:18.749 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:31:18.749 INFO:teuthology.orchestra.run.smithi094.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-04-25T11:31:18.770 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:31:18.794 INFO:teuthology.orchestra.run.smithi094.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.7 MB/s | 313 kB 00:00 2024-04-25T11:31:18.860 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:31:18.867 INFO:teuthology.orchestra.run.smithi094.stdout:(54/152): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-04-25T11:31:18.884 INFO:teuthology.orchestra.run.smithi094.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 205 kB/s | 18 kB 00:00 2024-04-25T11:31:18.900 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:31:18.909 INFO:teuthology.orchestra.run.smithi094.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 265 kB/s | 11 kB 00:00 2024-04-25T11:31:18.943 INFO:teuthology.orchestra.run.smithi094.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T11:31:18.960 INFO:teuthology.orchestra.run.smithi094.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-04-25T11:31:19.053 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T11:31:19.077 INFO:teuthology.orchestra.run.smithi094.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-04-25T11:31:19.105 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T11:31:19.110 INFO:teuthology.orchestra.run.smithi094.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-04-25T11:31:19.133 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:31:19.136 INFO:teuthology.orchestra.run.smithi094.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-04-25T11:31:19.161 INFO:teuthology.orchestra.run.smithi094.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 941 kB/s | 47 kB 00:00 2024-04-25T11:31:19.220 INFO:teuthology.orchestra.run.smithi094.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-25T11:31:19.236 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:31:19.254 INFO:teuthology.orchestra.run.smithi094.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-25T11:31:19.259 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:31:19.279 INFO:teuthology.orchestra.run.smithi094.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-04-25T11:31:19.321 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:31:19.343 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:31:19.363 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:31:19.384 INFO:teuthology.orchestra.run.smithi083.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 1.0 MB/s | 911 kB 00:00 2024-04-25T11:31:19.560 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:31:19.592 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:31:19.679 INFO:teuthology.orchestra.run.smithi094.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.2 MB/s | 3.0 MB 00:00 2024-04-25T11:31:19.699 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T11:31:19.794 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T11:31:19.801 INFO:teuthology.orchestra.run.smithi083.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 320 kB/s | 133 kB 00:00 2024-04-25T11:31:19.842 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T11:31:19.863 INFO:teuthology.orchestra.run.smithi094.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 7.8 MB/s | 4.7 MB 00:00 2024-04-25T11:31:19.876 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T11:31:19.955 INFO:teuthology.orchestra.run.smithi094.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-04-25T11:31:19.957 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T11:31:20.000 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T11:31:20.022 INFO:teuthology.orchestra.run.smithi094.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-04-25T11:31:20.044 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T11:31:20.047 INFO:teuthology.orchestra.run.smithi094.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-25T11:31:20.076 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T11:31:20.104 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T11:31:20.172 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T11:31:20.235 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T11:31:20.265 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T11:31:20.297 INFO:teuthology.orchestra.run.smithi094.stdout:(71/152): postgresql-13.14-1.el9.x86_64.rpm 2.6 MB/s | 1.6 MB 00:00 2024-04-25T11:31:20.300 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T11:31:20.373 INFO:teuthology.orchestra.run.smithi094.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-04-25T11:31:20.375 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T11:31:20.411 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T11:31:20.415 INFO:teuthology.orchestra.run.smithi094.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-04-25T11:31:20.454 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T11:31:20.483 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T11:31:20.524 INFO:teuthology.orchestra.run.smithi094.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-04-25T11:31:20.553 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T11:31:20.598 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T11:31:20.641 INFO:teuthology.orchestra.run.smithi094.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 9.3 MB/s | 5.8 MB 00:00 2024-04-25T11:31:20.672 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T11:31:20.674 INFO:teuthology.orchestra.run.smithi094.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-04-25T11:31:20.700 INFO:teuthology.orchestra.run.smithi094.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-04-25T11:31:20.725 INFO:teuthology.orchestra.run.smithi094.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-25T11:31:20.750 INFO:teuthology.orchestra.run.smithi094.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-25T11:31:20.776 INFO:teuthology.orchestra.run.smithi094.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-04-25T11:31:20.814 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T11:31:20.867 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T11:31:20.877 INFO:teuthology.orchestra.run.smithi083.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 1.3 MB/s | 4.7 MB 00:03 2024-04-25T11:31:20.909 INFO:teuthology.orchestra.run.smithi094.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-04-25T11:31:20.952 INFO:teuthology.orchestra.run.smithi083.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.2 MB/s | 4.4 MB 00:03 2024-04-25T11:31:21.177 INFO:teuthology.orchestra.run.smithi094.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-25T11:31:21.244 INFO:teuthology.orchestra.run.smithi094.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-25T11:31:21.269 INFO:teuthology.orchestra.run.smithi094.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 618 kB/s | 222 kB 00:00 2024-04-25T11:31:21.294 INFO:teuthology.orchestra.run.smithi083.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 256 kB/s | 87 kB 00:00 2024-04-25T11:31:21.311 INFO:teuthology.orchestra.run.smithi094.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 630 kB/s | 42 kB 00:00 2024-04-25T11:31:21.336 INFO:teuthology.orchestra.run.smithi094.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-04-25T11:31:21.392 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T11:31:21.403 INFO:teuthology.orchestra.run.smithi094.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-25T11:31:21.429 INFO:teuthology.orchestra.run.smithi094.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-04-25T11:31:21.471 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T11:31:21.500 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T11:31:21.504 INFO:teuthology.orchestra.run.smithi094.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 716 kB/s | 54 kB 00:00 2024-04-25T11:31:21.543 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T11:31:21.571 INFO:teuthology.orchestra.run.smithi094.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 304 kB/s | 51 kB 00:00 2024-04-25T11:31:21.581 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T11:31:21.622 INFO:teuthology.orchestra.run.smithi094.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 834 kB/s | 42 kB 00:00 2024-04-25T11:31:21.622 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T11:31:21.680 INFO:teuthology.orchestra.run.smithi094.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 756 kB/s | 44 kB 00:00 2024-04-25T11:31:21.785 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T11:31:21.786 INFO:teuthology.orchestra.run.smithi083.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-04-25T11:31:21.842 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T11:31:21.897 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T11:31:21.922 INFO:teuthology.orchestra.run.smithi094.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 914 kB/s | 221 kB 00:00 2024-04-25T11:31:21.969 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T11:31:21.972 INFO:teuthology.orchestra.run.smithi094.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-04-25T11:31:22.001 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T11:31:22.038 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T11:31:22.048 INFO:teuthology.orchestra.run.smithi094.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-04-25T11:31:22.084 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T11:31:22.120 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T11:31:22.163 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T11:31:22.186 INFO:teuthology.orchestra.run.smithi083.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 2.4 MB/s | 5.8 MB 00:02 2024-04-25T11:31:22.205 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T11:31:22.341 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T11:31:22.367 INFO:teuthology.orchestra.run.smithi083.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 424 kB/s | 244 kB 00:00 2024-04-25T11:31:22.382 INFO:teuthology.orchestra.run.smithi094.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 914 kB/s | 305 kB 00:00 2024-04-25T11:31:22.386 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T11:31:22.416 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T11:31:22.457 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T11:31:22.482 INFO:teuthology.orchestra.run.smithi094.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 578 kB/s | 58 kB 00:00 2024-04-25T11:31:22.552 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T11:31:22.582 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T11:31:22.622 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T11:31:22.628 INFO:teuthology.orchestra.run.smithi083.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 96 kB/s | 26 kB 00:00 2024-04-25T11:31:22.648 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T11:31:22.673 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T11:31:22.675 INFO:teuthology.orchestra.run.smithi094.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 488 kB/s | 94 kB 00:00 2024-04-25T11:31:22.678 INFO:teuthology.orchestra.run.smithi083.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 504 kB/s | 248 kB 00:00 2024-04-25T11:31:22.747 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T11:31:22.775 INFO:teuthology.orchestra.run.smithi094.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 636 kB/s | 64 kB 00:00 2024-04-25T11:31:22.842 INFO:teuthology.orchestra.run.smithi094.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 671 kB/s | 45 kB 00:00 2024-04-25T11:31:22.873 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T11:31:22.895 INFO:teuthology.orchestra.run.smithi083.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 72 kB/s | 19 kB 00:00 2024-04-25T11:31:22.922 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T11:31:22.950 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T11:31:22.984 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T11:31:23.023 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T11:31:23.054 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T11:31:23.085 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T11:31:23.117 INFO:teuthology.orchestra.run.smithi094.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-04-25T11:31:23.119 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T11:31:23.152 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T11:31:23.162 INFO:teuthology.orchestra.run.smithi083.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 130 kB/s | 35 kB 00:00 2024-04-25T11:31:23.194 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T11:31:23.228 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T11:31:23.262 INFO:teuthology.orchestra.run.smithi083.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 295 kB/s | 172 kB 00:00 2024-04-25T11:31:23.314 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T11:31:23.367 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T11:31:23.398 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T11:31:23.413 INFO:teuthology.orchestra.run.smithi083.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 2.8 MB/s | 6.0 MB 00:02 2024-04-25T11:31:23.445 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T11:31:23.551 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T11:31:23.568 INFO:teuthology.orchestra.run.smithi094.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.9 MB/s | 16 MB 00:02 2024-04-25T11:31:23.599 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T11:31:23.643 INFO:teuthology.orchestra.run.smithi094.stdout:(103/152): python3-babel-2.9.1-2.el9.noarch.rpm 1.8 MB/s | 6.0 MB 00:03 2024-04-25T11:31:23.852 INFO:teuthology.orchestra.run.smithi094.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-04-25T11:31:23.902 INFO:teuthology.orchestra.run.smithi094.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-04-25T11:31:23.936 INFO:teuthology.orchestra.run.smithi094.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-04-25T11:31:23.986 INFO:teuthology.orchestra.run.smithi094.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-04-25T11:31:23.996 INFO:teuthology.orchestra.run.smithi083.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 381 kB/s | 222 kB 00:00 2024-04-25T11:31:24.055 INFO:teuthology.orchestra.run.smithi083.stdout:(83/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 316 kB/s | 250 kB 00:00 2024-04-25T11:31:24.086 INFO:teuthology.orchestra.run.smithi094.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 485 kB/s | 49 kB 00:00 2024-04-25T11:31:24.270 INFO:teuthology.orchestra.run.smithi094.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-25T11:31:24.295 INFO:teuthology.orchestra.run.smithi094.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 323 kB/s | 67 kB 00:00 2024-04-25T11:31:24.321 INFO:teuthology.orchestra.run.smithi094.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-04-25T11:31:24.326 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T11:31:24.338 INFO:teuthology.orchestra.run.smithi083.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 226 kB/s | 77 kB 00:00 2024-04-25T11:31:24.371 INFO:teuthology.orchestra.run.smithi094.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-04-25T11:31:24.380 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T11:31:24.414 INFO:teuthology.orchestra.run.smithi083.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-04-25T11:31:24.414 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T11:31:24.421 INFO:teuthology.orchestra.run.smithi094.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 869 kB/s | 43 kB 00:00 2024-04-25T11:31:24.439 INFO:teuthology.orchestra.run.smithi083.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 110 kB/s | 42 kB 00:00 2024-04-25T11:31:24.464 INFO:teuthology.orchestra.run.smithi094.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-04-25T11:31:24.489 INFO:teuthology.orchestra.run.smithi094.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 480 kB/s | 32 kB 00:00 2024-04-25T11:31:24.514 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T11:31:24.514 INFO:teuthology.orchestra.run.smithi094.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-25T11:31:24.540 INFO:teuthology.orchestra.run.smithi094.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-25T11:31:24.571 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T11:31:24.590 INFO:teuthology.orchestra.run.smithi094.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-04-25T11:31:24.615 INFO:teuthology.orchestra.run.smithi094.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-25T11:31:24.641 INFO:teuthology.orchestra.run.smithi094.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 700 kB/s | 35 kB 00:00 2024-04-25T11:31:24.659 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T11:31:24.666 INFO:teuthology.orchestra.run.smithi094.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-25T11:31:24.691 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T11:31:24.717 INFO:teuthology.orchestra.run.smithi094.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-04-25T11:31:24.767 INFO:teuthology.orchestra.run.smithi094.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-04-25T11:31:24.784 INFO:teuthology.orchestra.run.smithi094.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-04-25T11:31:24.818 INFO:teuthology.orchestra.run.smithi094.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-25T11:31:24.843 INFO:teuthology.orchestra.run.smithi094.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 394 kB/s | 23 kB 00:00 2024-04-25T11:31:24.864 INFO:teuthology.orchestra.run.smithi083.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 512 kB/s | 269 kB 00:00 2024-04-25T11:31:24.867 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T11:31:24.868 INFO:teuthology.orchestra.run.smithi094.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-04-25T11:31:24.893 INFO:teuthology.orchestra.run.smithi094.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-04-25T11:31:24.897 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T11:31:24.935 INFO:teuthology.orchestra.run.smithi094.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-04-25T11:31:24.961 INFO:teuthology.orchestra.run.smithi094.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 689 kB/s | 46 kB 00:00 2024-04-25T11:31:24.973 INFO:teuthology.orchestra.run.smithi083.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 522 kB/s | 279 kB 00:00 2024-04-25T11:31:24.974 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T11:31:24.986 INFO:teuthology.orchestra.run.smithi094.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-25T11:31:24.998 INFO:teuthology.orchestra.run.smithi083.stdout:(89/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 272 kB/s | 159 kB 00:00 2024-04-25T11:31:25.020 INFO:teuthology.orchestra.run.smithi094.stdout:(132/152): protobuf-compiler-3.14.0-13.el9.x86_ 454 kB/s | 863 kB 00:01 2024-04-25T11:31:25.045 INFO:teuthology.orchestra.run.smithi094.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 687 kB/s | 58 kB 00:00 2024-04-25T11:31:25.071 INFO:teuthology.orchestra.run.smithi094.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-04-25T11:31:25.096 INFO:teuthology.orchestra.run.smithi094.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-25T11:31:25.146 INFO:teuthology.orchestra.run.smithi094.stdout:(136/152): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-04-25T11:31:25.197 INFO:teuthology.orchestra.run.smithi094.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 597 kB/s | 90 kB 00:00 2024-04-25T11:31:25.199 INFO:teuthology.orchestra.run.smithi083.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 153 kB/s | 51 kB 00:00 2024-04-25T11:31:25.230 INFO:teuthology.orchestra.run.smithi094.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-25T11:31:25.256 INFO:teuthology.orchestra.run.smithi094.stdout:(139/152): python3-pyperclip-1.8.2-2.el9.noarch 153 kB/s | 28 kB 00:00 2024-04-25T11:31:25.281 INFO:teuthology.orchestra.run.smithi094.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 704 kB/s | 59 kB 00:00 2024-04-25T11:31:25.306 INFO:teuthology.orchestra.run.smithi094.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 472 kB/s | 36 kB 00:00 2024-04-25T11:31:25.307 INFO:teuthology.orchestra.run.smithi083.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 161 kB/s | 54 kB 00:00 2024-04-25T11:31:25.332 INFO:teuthology.orchestra.run.smithi094.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 4.4 MB/s | 230 kB 00:00 2024-04-25T11:31:25.374 INFO:teuthology.orchestra.run.smithi094.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-25T11:31:25.399 INFO:teuthology.orchestra.run.smithi094.stdout:(144/152): python3-typing-extensions-4.4.0-2.el 356 kB/s | 51 kB 00:00 2024-04-25T11:31:25.425 INFO:teuthology.orchestra.run.smithi094.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-04-25T11:31:25.450 INFO:teuthology.orchestra.run.smithi094.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-25T11:31:25.457 INFO:teuthology.orchestra.run.smithi083.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 161 kB/s | 42 kB 00:00 2024-04-25T11:31:25.484 INFO:teuthology.orchestra.run.smithi094.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-04-25T11:31:25.584 INFO:teuthology.orchestra.run.smithi094.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-25T11:31:25.609 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T11:31:25.649 INFO:teuthology.orchestra.run.smithi083.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 129 kB/s | 44 kB 00:00 2024-04-25T11:31:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T11:31:25.801 INFO:teuthology.orchestra.run.smithi094.stdout:(149/152): librados2-19.0.0-3156.gd69f13d8.el7. 9.7 MB/s | 3.4 MB 00:00 2024-04-25T11:31:25.908 INFO:teuthology.orchestra.run.smithi083.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 139 kB/s | 36 kB 00:00 2024-04-25T11:31:25.958 INFO:teuthology.orchestra.run.smithi083.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 441 kB/s | 221 kB 00:00 2024-04-25T11:31:26.250 INFO:teuthology.orchestra.run.smithi083.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 139 kB/s | 47 kB 00:00 2024-04-25T11:31:26.443 INFO:teuthology.orchestra.run.smithi094.stdout:(150/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 3.3 MB/s | 3.1 MB 00:00 2024-04-25T11:31:26.534 INFO:teuthology.orchestra.run.smithi083.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 529 kB/s | 305 kB 00:00 2024-04-25T11:31:26.593 INFO:teuthology.orchestra.run.smithi083.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 169 kB/s | 58 kB 00:00 2024-04-25T11:31:26.635 INFO:teuthology.orchestra.run.smithi094.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-04-25T11:31:26.941 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T11:31:26.943 INFO:teuthology.orchestra.run.smithi083.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 182 kB/s | 64 kB 00:00 2024-04-25T11:31:26.968 INFO:teuthology.orchestra.run.smithi083.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 216 kB/s | 94 kB 00:00 2024-04-25T11:31:27.043 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T11:31:27.070 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T11:31:27.104 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T11:31:27.127 INFO:teuthology.orchestra.run.smithi083.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2024-04-25T11:31:27.294 INFO:teuthology.orchestra.run.smithi083.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 128 kB/s | 45 kB 00:00 2024-04-25T11:31:27.386 INFO:teuthology.orchestra.run.smithi094.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.5 MB/s | 2.4 MB 00:01 2024-04-25T11:31:27.388 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:31:27.388 INFO:teuthology.orchestra.run.smithi094.stdout:Total 9.6 MB/s | 222 MB 00:23 2024-04-25T11:31:27.431 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T11:31:27.484 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:31:27.570 INFO:teuthology.orchestra.run.smithi083.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-04-25T11:31:27.573 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:31:27.605 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T11:31:27.641 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T11:31:27.886 INFO:teuthology.orchestra.run.smithi083.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 926 kB/s | 548 kB 00:00 2024-04-25T11:31:27.970 INFO:teuthology.orchestra.run.smithi083.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 234 kB/s | 19 kB 00:00 2024-04-25T11:31:28.053 INFO:teuthology.orchestra.run.smithi083.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 639 kB/s | 309 kB 00:00 2024-04-25T11:31:28.117 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T11:31:28.145 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:31:28.145 INFO:teuthology.orchestra.run.smithi083.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-04-25T11:31:28.224 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:31:28.245 INFO:teuthology.orchestra.run.smithi083.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-04-25T11:31:28.246 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T11:31:28.362 INFO:teuthology.orchestra.run.smithi083.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-25T11:31:28.387 INFO:teuthology.orchestra.run.smithi083.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 474 kB/s | 67 kB 00:00 2024-04-25T11:31:28.438 INFO:teuthology.orchestra.run.smithi083.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-25T11:31:28.494 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-25T11:31:28.597 INFO:teuthology.orchestra.run.smithi083.stdout:(112/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.5 MB/s | 16 MB 00:03 2024-04-25T11:31:28.630 INFO:teuthology.orchestra.run.smithi083.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-04-25T11:31:28.635 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-25T11:31:28.635 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-25T11:31:28.672 INFO:teuthology.orchestra.run.smithi083.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 838 kB 00:00 2024-04-25T11:31:28.723 INFO:teuthology.orchestra.run.smithi083.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-04-25T11:31:28.764 INFO:teuthology.orchestra.run.smithi083.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 232 kB/s | 15 kB 00:00 2024-04-25T11:31:28.798 INFO:teuthology.orchestra.run.smithi083.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-25T11:31:28.856 INFO:teuthology.orchestra.run.smithi083.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.9 MB/s | 358 kB 00:00 2024-04-25T11:31:28.882 INFO:teuthology.orchestra.run.smithi083.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-25T11:31:28.915 INFO:teuthology.orchestra.run.smithi083.stdout:(120/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 136 kB/s | 43 kB 00:00 2024-04-25T11:31:28.941 INFO:teuthology.orchestra.run.smithi083.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 421 kB/s | 35 kB 00:00 2024-04-25T11:31:28.966 INFO:teuthology.orchestra.run.smithi083.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-25T11:31:29.016 INFO:teuthology.orchestra.run.smithi083.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-25T11:31:29.033 INFO:teuthology.orchestra.run.smithi083.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 159 kB/s | 11 kB 00:00 2024-04-25T11:31:29.083 INFO:teuthology.orchestra.run.smithi083.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 265 kB/s | 18 kB 00:00 2024-04-25T11:31:29.101 INFO:teuthology.orchestra.run.smithi083.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 345 kB/s | 23 kB 00:00 2024-04-25T11:31:29.143 INFO:teuthology.orchestra.run.smithi083.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 330 kB/s | 19 kB 00:00 2024-04-25T11:31:29.160 INFO:teuthology.orchestra.run.smithi083.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-04-25T11:31:29.227 INFO:teuthology.orchestra.run.smithi083.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-25T11:31:29.252 INFO:teuthology.orchestra.run.smithi083.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 504 kB/s | 46 kB 00:00 2024-04-25T11:31:29.311 INFO:teuthology.orchestra.run.smithi083.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 944 kB/s | 79 kB 00:00 2024-04-25T11:31:29.336 INFO:teuthology.orchestra.run.smithi083.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-04-25T11:31:29.378 INFO:teuthology.orchestra.run.smithi083.stdout:(133/152): python3-grpcio-1.46.7-10.el9.x86_64. 4.4 MB/s | 2.0 MB 00:00 2024-04-25T11:31:29.403 INFO:teuthology.orchestra.run.smithi083.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-25T11:31:29.428 INFO:teuthology.orchestra.run.smithi083.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-04-25T11:31:29.453 INFO:teuthology.orchestra.run.smithi083.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T11:31:29.479 INFO:teuthology.orchestra.run.smithi083.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-25T11:31:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-25T11:31:29.529 INFO:teuthology.orchestra.run.smithi083.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-25T11:31:29.555 INFO:teuthology.orchestra.run.smithi083.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-04-25T11:31:29.580 INFO:teuthology.orchestra.run.smithi083.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-04-25T11:31:29.605 INFO:teuthology.orchestra.run.smithi083.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 674 kB/s | 51 kB 00:00 2024-04-25T11:31:29.631 INFO:teuthology.orchestra.run.smithi083.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T11:31:29.656 INFO:teuthology.orchestra.run.smithi083.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T11:31:29.681 INFO:teuthology.orchestra.run.smithi083.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-04-25T11:31:29.706 INFO:teuthology.orchestra.run.smithi083.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-25T11:31:29.732 INFO:teuthology.orchestra.run.smithi083.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-25T11:31:29.790 INFO:teuthology.orchestra.run.smithi083.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-25T11:31:30.024 INFO:teuthology.orchestra.run.smithi083.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.8 MB/s | 1.3 MB 00:00 2024-04-25T11:31:30.083 INFO:teuthology.orchestra.run.smithi083.stdout:(150/152): librados2-19.0.0-3156.gd69f13d8.el7. 9.0 MB/s | 3.4 MB 00:00 2024-04-25T11:31:30.149 INFO:teuthology.orchestra.run.smithi083.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-04-25T11:31:30.463 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:31:30.493 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:31:30.636 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-25T11:31:30.636 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-25T11:31:31.024 INFO:teuthology.orchestra.run.smithi083.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.4 MB/s | 3.1 MB 00:01 2024-04-25T11:31:31.030 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:31:31.030 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:31:31.031 INFO:teuthology.orchestra.run.smithi083.stdout:Total 5.7 MB/s | 222 MB 00:38 2024-04-25T11:31:31.071 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:31:31.153 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:31:31.154 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T11:31:31.154 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T11:31:31.154 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:31:31.180 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:31:32.128 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-04-25T11:31:32.271 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-04-25T11:31:32.271 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-04-25T11:31:34.047 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-04-25T11:31:34.048 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-04-25T11:31:36.862 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-25T11:31:36.948 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T11:31:36.986 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T11:31:37.039 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T11:31:37.082 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T11:31:37.117 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T11:31:37.159 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T11:31:37.541 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T11:31:37.602 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:31:37.731 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:31:37.769 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:31:37.860 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:31:37.899 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:31:38.044 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T11:31:38.096 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T11:31:38.133 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:31:38.219 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:31:38.251 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:31:38.313 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:31:38.344 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:31:38.362 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:31:38.553 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:31:38.585 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:31:38.701 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T11:31:38.795 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T11:31:38.826 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T11:31:38.861 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T11:31:38.926 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T11:31:38.969 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T11:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T11:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T11:31:39.098 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T11:31:39.175 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T11:31:39.228 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T11:31:39.260 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T11:31:39.295 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T11:31:39.378 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T11:31:39.415 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T11:31:39.458 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T11:31:39.487 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T11:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T11:31:39.602 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T11:31:39.676 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T11:31:39.818 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T11:31:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T11:31:39.967 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-04-25T11:31:40.078 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T11:31:40.116 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T11:31:40.170 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T11:31:40.204 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T11:31:40.229 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T11:31:40.271 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T11:31:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T11:31:40.466 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T11:31:40.505 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T11:31:40.548 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T11:31:40.586 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T11:31:40.594 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T11:31:40.628 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T11:31:40.705 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:31:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T11:31:40.827 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:31:40.865 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:31:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T11:31:40.927 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T11:31:40.956 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:31:40.994 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:31:41.001 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T11:31:41.032 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T11:31:41.069 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T11:31:41.116 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T11:31:41.139 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T11:31:41.152 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T11:31:41.191 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T11:31:41.195 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T11:31:41.219 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:31:41.238 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T11:31:41.306 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:31:41.329 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:31:41.374 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T11:31:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T11:31:41.396 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:31:41.420 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:31:41.452 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T11:31:41.549 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T11:31:41.563 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:31:41.590 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T11:31:41.595 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:31:41.685 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T11:31:41.703 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T11:31:41.716 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T11:31:41.756 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T11:31:41.782 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T11:31:41.789 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T11:31:41.806 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T11:31:41.820 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T11:31:41.854 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T11:31:41.890 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T11:31:41.921 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T11:31:41.971 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T11:31:42.015 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T11:31:42.026 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T11:31:42.047 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T11:31:42.065 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T11:31:42.075 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T11:31:42.093 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T11:31:42.128 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T11:31:42.145 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T11:31:42.167 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T11:31:42.198 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T11:31:42.206 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T11:31:42.228 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T11:31:42.237 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T11:31:42.264 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T11:31:42.297 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T11:31:42.298 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T11:31:42.346 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T11:31:42.374 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T11:31:42.383 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T11:31:42.425 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T11:31:42.468 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T11:31:42.497 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T11:31:42.556 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:31:42.556 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /sys 2024-04-25T11:31:42.556 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /proc 2024-04-25T11:31:42.556 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /mnt 2024-04-25T11:31:42.557 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /var/tmp 2024-04-25T11:31:42.557 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /home 2024-04-25T11:31:42.557 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /root 2024-04-25T11:31:42.557 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /tmp 2024-04-25T11:31:42.557 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:31:42.566 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T11:31:42.597 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T11:31:42.611 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T11:31:42.685 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T11:31:42.695 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T11:31:42.726 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T11:31:42.735 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:31:42.753 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:31:42.753 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:31:42.753 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T11:31:42.753 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:31:42.773 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T11:31:42.827 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T11:31:42.881 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T11:31:42.888 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T11:31:42.936 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T11:31:43.420 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T11:31:43.517 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T11:31:43.555 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T11:31:43.598 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T11:31:43.636 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T11:31:43.648 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T11:31:43.677 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T11:31:43.701 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T11:31:43.735 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T11:31:43.832 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T11:31:43.842 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T11:31:43.893 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T11:31:43.905 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T11:31:43.960 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T11:31:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T11:31:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T11:31:44.032 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T11:31:44.064 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T11:31:44.092 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T11:31:44.140 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T11:31:44.175 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T11:31:44.181 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T11:31:44.218 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T11:31:44.228 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T11:31:44.230 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:31:44.255 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:31:44.255 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:31:44.255 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T11:31:44.255 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:31:44.255 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:31:44.255 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:31:44.260 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T11:31:44.305 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T11:31:44.396 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T11:31:44.441 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T11:31:44.473 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T11:31:44.489 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:31:44.511 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:31:44.511 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:31:44.511 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T11:31:44.511 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:31:44.511 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:31:44.511 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:31:44.513 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T11:31:44.606 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T11:31:44.646 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T11:31:44.686 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T11:31:44.712 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T11:31:44.737 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T11:31:44.810 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T11:31:44.937 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T11:31:44.948 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:31:44.952 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T11:31:44.969 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T11:31:44.970 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:31:44.970 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:31:44.970 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T11:31:44.970 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:31:44.970 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:31:44.970 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:31:44.998 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T11:31:45.031 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T11:31:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T11:31:45.070 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T11:31:45.078 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T11:31:45.102 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T11:31:45.119 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T11:31:45.132 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T11:31:45.173 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T11:31:45.183 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T11:31:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T11:31:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T11:31:45.292 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T11:31:45.361 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T11:31:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T11:31:45.407 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T11:31:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T11:31:45.462 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T11:31:45.477 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T11:31:45.517 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T11:31:45.566 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:31:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T11:31:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T11:31:45.705 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:31:45.741 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T11:31:45.773 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T11:31:45.800 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T11:31:46.252 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:31:46.282 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:31:46.290 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T11:31:46.314 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T11:31:46.369 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T11:31:46.376 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T11:31:46.382 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T11:31:46.397 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T11:31:46.405 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:31:46.436 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T11:31:46.447 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T11:31:46.470 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T11:31:46.566 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T11:31:46.619 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T11:31:46.697 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T11:31:46.738 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T11:31:46.759 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T11:31:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:31:46.876 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:31:46.907 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T11:31:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T11:31:46.936 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T11:31:46.943 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T11:31:47.013 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T11:31:47.424 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T11:31:47.444 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:31:47.464 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:31:47.482 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:31:47.535 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:31:47.565 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T11:31:47.646 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T11:31:47.758 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T11:31:48.442 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:31:48.474 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:31:48.577 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:31:48.739 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T11:31:48.768 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:31:48.791 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:31:48.791 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:31:48.791 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T11:31:48.791 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:31:48.791 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:31:48.791 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:31:48.835 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:31:48.846 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:31:48.943 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T11:31:48.972 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T11:31:49.000 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T11:31:49.075 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T11:31:49.102 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T11:31:49.135 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T11:31:49.447 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T11:31:49.490 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:31:49.581 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T11:31:49.611 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T11:31:49.648 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T11:31:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:31:49.838 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:31:49.929 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T11:31:49.963 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:31:49.985 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:31:49.986 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:31:49.986 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T11:31:49.986 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:31:49.986 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:31:49.986 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:31:50.030 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:31:50.052 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:31:50.052 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:31:50.052 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T11:31:50.052 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:31:50.097 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:31:50.120 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:31:50.120 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:31:50.120 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T11:31:50.120 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:31:50.120 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:31:50.120 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:31:50.149 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T11:31:50.185 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:31:50.256 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T11:31:50.278 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T11:31:50.377 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:31:50.417 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:31:50.423 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:31:50.445 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:31:50.446 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:31:50.446 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T11:31:50.446 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:31:50.446 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:31:50.446 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:31:50.507 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:31:50.507 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T11:31:50.507 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T11:31:50.507 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T11:31:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:31:52.526 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:31:52.566 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:31:53.112 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T11:31:53.145 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:31:53.229 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T11:31:53.229 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T11:31:53.229 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T11:31:53.229 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:31:53.254 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:31:55.052 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T11:31:55.093 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T11:31:55.142 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T11:31:55.183 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T11:31:55.227 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T11:31:55.414 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T11:31:55.457 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T11:31:55.486 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T11:31:56.044 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:31:56.084 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:31:56.104 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:31:56.938 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T11:31:57.022 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T11:31:57.063 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T11:31:57.155 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T11:31:57.193 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T11:31:57.193 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:31:57.277 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:31:57.310 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:31:57.311 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:31:57.404 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:31:57.475 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:32:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:32:02.050 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /sys 2024-04-25T11:32:02.050 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /proc 2024-04-25T11:32:02.050 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /mnt 2024-04-25T11:32:02.050 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /var/tmp 2024-04-25T11:32:02.050 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /home 2024-04-25T11:32:02.050 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /root 2024-04-25T11:32:02.050 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /tmp 2024-04-25T11:32:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T11:32:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:32:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:32:02.238 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:02.238 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T11:32:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T11:32:03.611 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:32:03.612 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T11:32:03.612 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T11:32:03.612 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T11:32:03.612 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T11:32:03.612 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T11:32:03.612 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T11:32:03.612 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T11:32:03.612 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:32:03.612 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:32:03.612 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:32:03.612 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T11:32:03.612 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T11:32:03.612 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:32:03.613 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T11:32:03.613 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T11:32:03.613 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T11:32:03.613 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T11:32:03.613 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T11:32:03.613 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T11:32:03.613 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T11:32:03.613 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T11:32:03.613 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T11:32:03.613 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T11:32:03.613 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T11:32:03.613 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T11:32:03.613 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T11:32:03.613 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T11:32:03.614 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T11:32:03.614 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T11:32:03.614 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T11:32:03.625 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T11:32:03.626 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T11:32:03.626 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T11:32:03.626 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T11:32:03.626 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T11:32:03.626 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T11:32:03.626 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T11:32:03.626 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T11:32:03.626 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T11:32:03.626 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T11:32:03.626 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T11:32:03.626 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T11:32:03.627 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T11:32:03.627 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T11:32:03.627 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T11:32:03.627 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T11:32:03.627 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T11:32:03.627 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T11:32:03.627 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T11:32:03.627 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T11:32:03.627 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T11:32:03.627 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T11:32:03.627 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T11:32:03.627 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T11:32:03.627 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T11:32:03.628 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T11:32:03.628 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T11:32:03.628 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T11:32:03.628 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T11:32:03.628 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T11:32:03.628 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T11:32:03.628 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T11:32:03.628 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T11:32:03.628 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T11:32:03.628 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T11:32:03.628 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T11:32:03.628 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T11:32:03.628 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T11:32:03.629 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T11:32:03.629 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T11:32:03.629 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T11:32:03.629 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T11:32:03.629 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T11:32:03.629 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T11:32:03.629 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T11:32:03.629 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T11:32:03.629 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T11:32:03.629 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T11:32:03.629 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T11:32:03.629 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T11:32:03.629 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T11:32:03.630 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T11:32:03.630 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T11:32:03.630 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T11:32:03.630 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T11:32:03.630 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T11:32:03.630 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T11:32:03.630 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T11:32:03.630 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T11:32:03.630 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T11:32:03.630 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T11:32:03.630 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T11:32:03.630 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T11:32:03.630 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T11:32:03.631 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T11:32:03.631 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T11:32:03.631 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T11:32:03.631 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T11:32:03.631 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T11:32:03.631 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T11:32:03.631 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T11:32:03.631 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T11:32:03.631 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T11:32:03.631 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T11:32:03.631 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T11:32:03.631 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T11:32:03.631 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T11:32:03.631 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T11:32:03.632 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T11:32:03.632 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T11:32:03.632 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T11:32:03.632 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T11:32:03.632 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T11:32:03.632 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T11:32:03.632 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T11:32:03.632 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T11:32:03.632 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T11:32:03.632 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:32:03.632 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:32:03.632 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T11:32:03.632 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T11:32:03.633 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T11:32:03.633 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T11:32:03.633 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T11:32:03.633 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T11:32:03.633 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T11:32:03.633 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T11:32:03.633 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T11:32:03.633 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T11:32:03.633 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T11:32:03.633 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T11:32:03.633 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T11:32:03.633 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T11:32:03.633 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T11:32:03.633 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T11:32:03.634 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T11:32:03.634 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T11:32:03.634 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T11:32:03.634 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T11:32:03.634 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T11:32:03.634 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T11:32:03.634 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T11:32:03.634 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T11:32:03.634 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T11:32:03.634 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T11:32:03.634 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T11:32:03.634 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T11:32:03.635 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T11:32:03.635 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T11:32:03.635 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T11:32:03.635 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T11:32:03.635 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T11:32:03.635 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T11:32:03.635 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:32:03.635 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T11:32:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:32:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:32:03.762 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:03.762 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T11:32:03.762 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:32:03.762 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:32:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T11:32:03.986 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:32:04.008 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:32:04.009 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:04.009 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T11:32:04.009 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:32:04.009 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:32:04.009 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T11:32:04.453 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:32:04.476 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:32:04.476 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:04.476 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T11:32:04.476 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:32:04.476 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:32:04.476 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T11:32:04.584 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T11:32:04.640 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T11:32:04.668 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T11:32:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2024-04-25T11:32:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2024-04-25T11:32:04.669 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2024-04-25T11:32:04.669 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2024-04-25T11:32:04.669 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2024-04-25T11:32:04.669 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /root 2024-04-25T11:32:04.669 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2024-04-25T11:32:04.669 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:32:04.696 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T11:32:04.843 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:32:04.859 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T11:32:04.859 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:04.859 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T11:32:04.859 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:32:04.883 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T11:32:04.938 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T11:32:05.017 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T11:32:05.105 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:32:05.236 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:32:05.281 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T11:32:05.313 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T11:32:05.341 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T11:32:05.768 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:32:05.768 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:32:05.768 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2024-04-25T11:32:05.768 INFO:teuthology.orchestra.run.smithi111.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.768 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.768 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T11:32:05.769 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T11:32:05.769 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-04-25T11:32:05.769 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T11:32:05.769 INFO:teuthology.orchestra.run.smithi111.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T11:32:05.769 INFO:teuthology.orchestra.run.smithi111.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T11:32:05.769 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T11:32:05.769 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.769 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.769 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.769 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.769 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:05.769 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.770 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.770 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.770 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:05.770 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:05.770 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:05.770 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:05.770 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:05.770 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.770 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.770 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:05.770 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.770 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.771 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.771 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:05.771 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:05.771 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.771 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.771 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:05.771 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T11:32:05.771 INFO:teuthology.orchestra.run.smithi111.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T11:32:05.771 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T11:32:05.771 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T11:32:05.771 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T11:32:05.771 INFO:teuthology.orchestra.run.smithi111.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T11:32:05.772 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T11:32:05.772 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T11:32:05.772 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T11:32:05.772 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T11:32:05.772 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T11:32:05.772 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.772 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.772 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.772 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T11:32:05.772 INFO:teuthology.orchestra.run.smithi111.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T11:32:05.772 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T11:32:05.772 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T11:32:05.772 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T11:32:05.772 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.773 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.773 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T11:32:05.773 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.773 INFO:teuthology.orchestra.run.smithi111.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T11:32:05.773 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T11:32:05.773 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T11:32:05.773 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T11:32:05.773 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T11:32:05.773 INFO:teuthology.orchestra.run.smithi111.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T11:32:05.773 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T11:32:05.773 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T11:32:05.773 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T11:32:05.773 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T11:32:05.774 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T11:32:05.774 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T11:32:05.774 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T11:32:05.774 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T11:32:05.774 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T11:32:05.774 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T11:32:05.774 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T11:32:05.774 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T11:32:05.774 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T11:32:05.774 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T11:32:05.774 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T11:32:05.774 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T11:32:05.774 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T11:32:05.774 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T11:32:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T11:32:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T11:32:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T11:32:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T11:32:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T11:32:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T11:32:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T11:32:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T11:32:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T11:32:05.775 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T11:32:05.776 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T11:32:05.776 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T11:32:05.776 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T11:32:05.776 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T11:32:05.776 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T11:32:05.776 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T11:32:05.776 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T11:32:05.776 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T11:32:05.776 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T11:32:05.776 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T11:32:05.776 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T11:32:05.776 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T11:32:05.776 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T11:32:05.777 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T11:32:05.777 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T11:32:05.777 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T11:32:05.777 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T11:32:05.777 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T11:32:05.777 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T11:32:05.777 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T11:32:05.777 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T11:32:05.777 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T11:32:05.777 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T11:32:05.777 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T11:32:05.777 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T11:32:05.777 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T11:32:05.778 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T11:32:05.778 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T11:32:05.778 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T11:32:05.778 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.778 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.778 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T11:32:05.778 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T11:32:05.778 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T11:32:05.778 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.778 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T11:32:05.778 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T11:32:05.778 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T11:32:05.778 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T11:32:05.778 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T11:32:05.779 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T11:32:05.779 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T11:32:05.779 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T11:32:05.779 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T11:32:05.779 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T11:32:05.779 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T11:32:05.779 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T11:32:05.779 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T11:32:05.779 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T11:32:05.779 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T11:32:05.779 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.779 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.779 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:05.779 INFO:teuthology.orchestra.run.smithi111.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T11:32:05.780 INFO:teuthology.orchestra.run.smithi111.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T11:32:05.780 INFO:teuthology.orchestra.run.smithi111.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T11:32:05.780 INFO:teuthology.orchestra.run.smithi111.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T11:32:05.780 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T11:32:05.780 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T11:32:05.780 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T11:32:05.780 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:32:05.780 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-04-25T11:32:05.793 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:32:05.824 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:32:05.856 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T11:32:05.918 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T11:32:05.947 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:32:06.134 DEBUG:teuthology.parallel:result is None 2024-04-25T11:32:06.347 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:32:06.373 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T11:32:06.373 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:06.373 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T11:32:06.373 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:32:06.373 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:32:06.373 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:32:06.598 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:32:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T11:32:06.620 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:06.620 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T11:32:06.620 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:32:06.620 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:32:06.620 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:32:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:32:07.024 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:32:07.065 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:32:07.087 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T11:32:07.087 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:07.087 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T11:32:07.087 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:32:07.088 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:32:07.088 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:32:07.186 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T11:32:07.235 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T11:32:07.291 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T11:32:07.475 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T11:32:07.524 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T11:32:07.593 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T11:32:07.682 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:32:07.821 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:32:07.866 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T11:32:07.898 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T11:32:07.934 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T11:32:07.984 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:32:08.008 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:32:08.272 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T11:32:08.310 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:32:08.333 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:32:08.333 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:08.333 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T11:32:08.333 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:32:08.334 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:32:08.334 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T11:32:08.352 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:32:08.378 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:32:08.383 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T11:32:08.389 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:32:08.415 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T11:32:08.476 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T11:32:08.487 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T11:32:08.514 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:32:08.535 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T11:32:09.457 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T11:32:09.498 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:32:09.521 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:32:09.521 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:09.521 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T11:32:09.521 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:32:09.521 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:32:09.521 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T11:32:09.549 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T11:32:09.573 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:32:09.591 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:32:09.595 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:32:09.595 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:09.596 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T11:32:09.596 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T11:32:09.649 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:32:09.672 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:32:09.672 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:09.672 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T11:32:09.672 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:32:09.672 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:32:09.672 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T11:32:09.977 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:32:10.000 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:32:10.000 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:10.000 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T11:32:10.000 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:32:10.000 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:32:10.000 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T11:32:10.559 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T11:32:10.583 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:32:10.683 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T11:32:10.846 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T11:32:10.885 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:32:10.908 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T11:32:10.908 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:10.908 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T11:32:10.908 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:32:10.909 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:32:10.909 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:32:10.961 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:32:10.972 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T11:32:11.070 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T11:32:11.118 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T11:32:12.040 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T11:32:12.072 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:32:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T11:32:12.095 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:12.095 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T11:32:12.095 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:32:12.095 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:32:12.095 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:32:12.140 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:32:12.161 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T11:32:12.162 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:12.162 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T11:32:12.162 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:32:12.207 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:32:12.229 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T11:32:12.229 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:12.229 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T11:32:12.229 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:32:12.229 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:32:12.229 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:32:12.532 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:32:12.554 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T11:32:12.554 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:32:12.555 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T11:32:12.555 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:32:12.555 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:32:12.555 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:32:14.580 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T11:32:14.622 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T11:32:14.662 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T11:32:14.703 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T11:32:14.730 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T11:32:14.919 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T11:32:14.959 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T11:32:14.989 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T11:32:15.633 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:32:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:32:16.521 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T11:32:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T11:32:16.648 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T11:32:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T11:32:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T11:32:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:32:16.862 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:32:16.895 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:32:16.895 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:32:16.982 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:32:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:32:17.145 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T11:32:17.186 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T11:32:17.226 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T11:32:17.259 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T11:32:17.286 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T11:32:17.472 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T11:32:17.516 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T11:32:17.546 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T11:32:18.133 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:32:18.177 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:32:18.197 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:32:19.026 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T11:32:19.133 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T11:32:19.174 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T11:32:19.255 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T11:32:19.287 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T11:32:19.287 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:32:19.390 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:32:19.429 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:32:19.429 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:32:19.524 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:32:19.594 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:32:23.045 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:32:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T11:32:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T11:32:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T11:32:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T11:32:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T11:32:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T11:32:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T11:32:23.046 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:32:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:32:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:32:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T11:32:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T11:32:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:32:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T11:32:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T11:32:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T11:32:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T11:32:23.047 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T11:32:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T11:32:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T11:32:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T11:32:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T11:32:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T11:32:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T11:32:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T11:32:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T11:32:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T11:32:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T11:32:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T11:32:23.048 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T11:32:23.060 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T11:32:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T11:32:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T11:32:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T11:32:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T11:32:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T11:32:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T11:32:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T11:32:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T11:32:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T11:32:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T11:32:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T11:32:23.061 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T11:32:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T11:32:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T11:32:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T11:32:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T11:32:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T11:32:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T11:32:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T11:32:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T11:32:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T11:32:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T11:32:23.062 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T11:32:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T11:32:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T11:32:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T11:32:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T11:32:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T11:32:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T11:32:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T11:32:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T11:32:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T11:32:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T11:32:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T11:32:23.063 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T11:32:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T11:32:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T11:32:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T11:32:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T11:32:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T11:32:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T11:32:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T11:32:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T11:32:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T11:32:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T11:32:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T11:32:23.064 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T11:32:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T11:32:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T11:32:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T11:32:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T11:32:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T11:32:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T11:32:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T11:32:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T11:32:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T11:32:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T11:32:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T11:32:23.065 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T11:32:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T11:32:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T11:32:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T11:32:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T11:32:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T11:32:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T11:32:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T11:32:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T11:32:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T11:32:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T11:32:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T11:32:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T11:32:23.066 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T11:32:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T11:32:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T11:32:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T11:32:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T11:32:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T11:32:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T11:32:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T11:32:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T11:32:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T11:32:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T11:32:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T11:32:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T11:32:23.067 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T11:32:23.068 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T11:32:23.068 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:32:23.068 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:32:23.068 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T11:32:23.068 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T11:32:23.068 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T11:32:23.068 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T11:32:23.068 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T11:32:23.068 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T11:32:23.068 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T11:32:23.068 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T11:32:23.068 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T11:32:23.068 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T11:32:23.069 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T11:32:23.069 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T11:32:23.069 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T11:32:23.069 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T11:32:23.069 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T11:32:23.069 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T11:32:23.069 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T11:32:23.069 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T11:32:23.069 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T11:32:23.069 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T11:32:23.069 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T11:32:23.069 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T11:32:23.069 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T11:32:23.070 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T11:32:23.070 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T11:32:23.070 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T11:32:23.070 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T11:32:23.070 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T11:32:23.070 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T11:32:23.070 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T11:32:23.070 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T11:32:23.070 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T11:32:23.070 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T11:32:23.070 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T11:32:23.070 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:32:23.071 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T11:32:25.109 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:32:25.109 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T11:32:25.109 INFO:teuthology.orchestra.run.smithi094.stdout:Upgraded: 2024-04-25T11:32:25.109 INFO:teuthology.orchestra.run.smithi094.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.110 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.110 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T11:32:25.110 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T11:32:25.110 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-25T11:32:25.110 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T11:32:25.110 INFO:teuthology.orchestra.run.smithi094.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T11:32:25.110 INFO:teuthology.orchestra.run.smithi094.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T11:32:25.110 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T11:32:25.110 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.110 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.110 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.110 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.110 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:25.111 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.111 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.111 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.111 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:25.111 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:25.111 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:25.111 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:25.111 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:25.111 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.111 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.111 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:25.111 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.111 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.111 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.112 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:25.112 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:25.112 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.112 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.112 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:25.112 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T11:32:25.112 INFO:teuthology.orchestra.run.smithi094.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T11:32:25.112 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T11:32:25.112 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T11:32:25.112 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T11:32:25.112 INFO:teuthology.orchestra.run.smithi094.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T11:32:25.112 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T11:32:25.112 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T11:32:25.113 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T11:32:25.113 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T11:32:25.113 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T11:32:25.113 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.113 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.113 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.113 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T11:32:25.113 INFO:teuthology.orchestra.run.smithi094.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T11:32:25.113 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T11:32:25.113 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T11:32:25.113 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T11:32:25.113 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.113 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.113 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T11:32:25.114 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.114 INFO:teuthology.orchestra.run.smithi094.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T11:32:25.114 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T11:32:25.114 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T11:32:25.114 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T11:32:25.114 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T11:32:25.114 INFO:teuthology.orchestra.run.smithi094.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T11:32:25.114 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T11:32:25.114 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T11:32:25.114 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T11:32:25.114 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T11:32:25.114 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T11:32:25.114 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T11:32:25.115 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T11:32:25.115 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T11:32:25.115 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T11:32:25.115 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T11:32:25.115 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T11:32:25.115 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T11:32:25.115 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T11:32:25.115 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T11:32:25.115 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T11:32:25.115 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T11:32:25.115 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T11:32:25.115 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.115 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.115 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.116 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T11:32:25.116 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T11:32:25.116 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T11:32:25.116 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T11:32:25.116 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T11:32:25.116 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T11:32:25.116 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T11:32:25.116 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T11:32:25.116 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T11:32:25.116 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T11:32:25.116 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T11:32:25.116 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T11:32:25.116 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T11:32:25.116 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T11:32:25.117 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T11:32:25.117 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T11:32:25.117 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T11:32:25.117 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T11:32:25.117 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T11:32:25.117 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T11:32:25.117 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T11:32:25.117 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T11:32:25.117 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T11:32:25.117 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T11:32:25.117 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T11:32:25.117 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T11:32:25.117 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T11:32:25.117 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T11:32:25.118 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T11:32:25.118 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T11:32:25.118 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T11:32:25.118 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T11:32:25.118 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T11:32:25.118 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T11:32:25.118 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T11:32:25.118 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T11:32:25.118 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T11:32:25.118 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T11:32:25.118 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T11:32:25.118 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T11:32:25.118 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.119 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.119 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T11:32:25.119 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T11:32:25.119 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T11:32:25.119 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.119 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T11:32:25.119 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T11:32:25.119 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T11:32:25.119 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T11:32:25.119 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T11:32:25.119 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T11:32:25.119 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T11:32:25.119 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T11:32:25.119 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T11:32:25.120 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T11:32:25.120 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T11:32:25.120 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T11:32:25.120 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T11:32:25.120 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T11:32:25.120 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T11:32:25.120 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.120 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.120 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:25.120 INFO:teuthology.orchestra.run.smithi094.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T11:32:25.120 INFO:teuthology.orchestra.run.smithi094.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T11:32:25.120 INFO:teuthology.orchestra.run.smithi094.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T11:32:25.120 INFO:teuthology.orchestra.run.smithi094.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T11:32:25.120 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T11:32:25.121 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T11:32:25.121 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T11:32:25.121 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T11:32:25.121 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-25T11:32:25.510 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:32:25.510 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T11:32:25.510 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T11:32:25.510 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T11:32:25.510 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T11:32:25.510 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T11:32:25.510 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T11:32:25.510 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T11:32:25.510 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T11:32:25.511 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T11:32:25.511 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T11:32:25.511 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T11:32:25.511 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T11:32:25.511 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T11:32:25.511 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T11:32:25.511 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T11:32:25.511 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T11:32:25.511 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T11:32:25.511 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T11:32:25.511 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T11:32:25.511 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T11:32:25.511 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T11:32:25.512 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T11:32:25.512 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T11:32:25.512 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T11:32:25.512 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T11:32:25.512 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T11:32:25.512 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T11:32:25.512 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T11:32:25.512 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T11:32:25.512 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T11:32:25.524 DEBUG:teuthology.parallel:result is None 2024-04-25T11:32:25.527 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T11:32:25.527 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T11:32:25.528 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T11:32:25.528 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T11:32:25.528 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T11:32:25.528 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T11:32:25.528 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T11:32:25.528 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T11:32:25.528 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T11:32:25.528 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T11:32:25.528 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T11:32:25.528 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T11:32:25.528 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T11:32:25.528 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T11:32:25.528 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T11:32:25.529 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T11:32:25.529 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T11:32:25.529 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T11:32:25.529 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T11:32:25.529 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T11:32:25.529 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T11:32:25.529 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T11:32:25.529 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T11:32:25.529 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T11:32:25.529 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T11:32:25.529 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T11:32:25.529 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T11:32:25.529 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T11:32:25.529 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T11:32:25.529 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T11:32:25.530 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T11:32:25.530 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T11:32:25.530 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T11:32:25.530 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T11:32:25.530 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T11:32:25.530 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T11:32:25.530 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T11:32:25.530 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T11:32:25.530 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T11:32:25.530 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T11:32:25.530 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T11:32:25.530 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T11:32:25.530 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T11:32:25.531 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T11:32:25.531 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T11:32:25.531 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T11:32:25.531 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T11:32:25.531 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T11:32:25.531 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T11:32:25.531 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T11:32:25.531 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T11:32:25.531 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T11:32:25.531 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T11:32:25.531 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T11:32:25.531 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T11:32:25.531 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T11:32:25.531 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T11:32:25.532 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T11:32:25.532 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T11:32:25.532 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T11:32:25.532 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T11:32:25.532 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T11:32:25.532 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T11:32:25.532 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T11:32:25.532 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T11:32:25.532 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T11:32:25.532 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T11:32:25.532 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T11:32:25.532 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T11:32:25.532 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T11:32:25.533 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T11:32:25.533 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T11:32:25.533 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T11:32:25.533 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T11:32:25.533 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T11:32:25.533 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T11:32:25.533 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T11:32:25.533 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T11:32:25.533 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T11:32:25.533 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T11:32:25.533 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T11:32:25.533 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T11:32:25.533 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T11:32:25.534 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T11:32:25.534 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T11:32:25.534 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T11:32:25.534 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T11:32:25.534 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:32:25.534 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:32:25.534 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T11:32:25.534 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T11:32:25.534 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T11:32:25.534 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T11:32:25.534 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T11:32:25.534 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T11:32:25.534 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T11:32:25.535 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T11:32:25.535 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T11:32:25.535 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T11:32:25.535 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T11:32:25.535 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T11:32:25.535 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T11:32:25.535 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T11:32:25.535 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T11:32:25.535 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T11:32:25.535 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T11:32:25.535 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T11:32:25.535 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T11:32:25.535 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T11:32:25.535 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T11:32:25.536 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T11:32:25.536 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T11:32:25.536 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T11:32:25.536 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T11:32:25.536 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T11:32:25.536 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T11:32:25.536 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T11:32:25.536 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T11:32:25.536 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T11:32:25.536 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T11:32:25.536 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T11:32:25.536 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T11:32:25.536 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T11:32:25.536 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:32:25.536 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T11:32:27.661 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:32:27.662 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:32:27.662 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2024-04-25T11:32:27.662 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.662 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.662 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T11:32:27.662 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T11:32:27.662 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-04-25T11:32:27.662 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T11:32:27.662 INFO:teuthology.orchestra.run.smithi083.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T11:32:27.662 INFO:teuthology.orchestra.run.smithi083.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T11:32:27.662 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T11:32:27.662 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.663 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.663 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.663 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.663 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:27.663 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.663 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.663 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.663 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:27.663 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:27.663 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:27.663 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:27.663 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:27.663 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.663 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.664 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:27.664 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.664 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.664 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.664 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:27.664 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:27.664 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.664 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.664 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T11:32:27.664 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T11:32:27.664 INFO:teuthology.orchestra.run.smithi083.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T11:32:27.664 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T11:32:27.664 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T11:32:27.664 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T11:32:27.665 INFO:teuthology.orchestra.run.smithi083.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T11:32:27.665 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T11:32:27.665 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T11:32:27.665 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T11:32:27.665 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T11:32:27.665 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T11:32:27.665 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.665 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.665 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.665 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T11:32:27.665 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T11:32:27.665 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T11:32:27.665 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T11:32:27.665 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T11:32:27.665 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.666 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.666 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T11:32:27.666 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.666 INFO:teuthology.orchestra.run.smithi083.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T11:32:27.666 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T11:32:27.666 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T11:32:27.666 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T11:32:27.666 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T11:32:27.666 INFO:teuthology.orchestra.run.smithi083.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T11:32:27.666 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T11:32:27.666 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T11:32:27.666 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T11:32:27.666 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T11:32:27.666 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T11:32:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T11:32:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T11:32:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T11:32:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T11:32:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T11:32:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T11:32:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T11:32:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T11:32:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T11:32:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T11:32:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T11:32:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T11:32:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.667 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.668 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.668 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T11:32:27.668 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T11:32:27.668 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T11:32:27.668 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T11:32:27.668 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T11:32:27.668 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T11:32:27.668 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T11:32:27.668 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T11:32:27.668 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T11:32:27.668 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T11:32:27.668 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T11:32:27.668 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T11:32:27.669 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T11:32:27.669 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T11:32:27.669 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T11:32:27.669 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T11:32:27.669 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T11:32:27.669 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T11:32:27.669 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T11:32:27.669 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T11:32:27.669 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T11:32:27.669 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T11:32:27.669 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T11:32:27.669 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T11:32:27.670 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T11:32:27.670 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T11:32:27.670 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T11:32:27.670 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T11:32:27.670 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T11:32:27.670 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T11:32:27.670 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T11:32:27.670 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T11:32:27.670 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T11:32:27.670 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T11:32:27.670 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T11:32:27.670 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T11:32:27.671 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T11:32:27.671 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T11:32:27.671 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T11:32:27.671 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T11:32:27.671 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.671 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.671 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T11:32:27.671 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T11:32:27.671 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T11:32:27.671 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.671 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T11:32:27.671 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T11:32:27.672 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T11:32:27.672 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T11:32:27.672 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T11:32:27.672 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T11:32:27.672 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T11:32:27.672 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T11:32:27.672 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T11:32:27.672 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T11:32:27.672 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T11:32:27.672 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T11:32:27.672 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T11:32:27.672 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T11:32:27.673 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T11:32:27.673 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.673 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.673 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T11:32:27.673 INFO:teuthology.orchestra.run.smithi083.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T11:32:27.673 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T11:32:27.673 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T11:32:27.673 INFO:teuthology.orchestra.run.smithi083.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T11:32:27.673 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T11:32:27.673 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T11:32:27.674 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T11:32:27.674 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:32:27.674 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-04-25T11:32:28.047 DEBUG:teuthology.parallel:result is None 2024-04-25T11:32:28.048 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:32:28.261 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:32:28.282 INFO:teuthology.orchestra.run.smithi083.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T11:32:28.283 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T11:32:28.283 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T11:32:28.285 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:32:28.421 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:32:28.445 INFO:teuthology.orchestra.run.smithi094.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T11:32:28.446 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T11:32:28.446 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T11:32:28.447 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:32:28.581 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:32:28.607 INFO:teuthology.orchestra.run.smithi111.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T11:32:28.607 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T11:32:28.607 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T11:32:28.609 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T11:32:28.609 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:32:28.609 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:32:28.634 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T11:32:28.634 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:32:28.666 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-25T11:32:28.666 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:32:28.696 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T11:32:28.696 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:32:28.696 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:32:28.719 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:32:28.783 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T11:32:28.783 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:32:28.811 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:32:28.879 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-25T11:32:28.879 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:32:28.910 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:32:28.976 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T11:32:28.977 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:32:28.977 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:32:29.000 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:32:29.065 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T11:32:29.065 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:32:29.094 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:32:29.162 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-25T11:32:29.162 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:32:29.185 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:32:29.249 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T11:32:29.250 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:32:29.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:32:29.273 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:32:29.336 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T11:32:29.336 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:32:29.365 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:32:29.433 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-25T11:32:29.434 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:32:29.457 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:32:29.521 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T11:32:29.624 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2}} 2024-04-25T11:32:29.624 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:32:29.624 INFO:tasks.cephadm:Cluster fsid is 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:32:29.624 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T11:32:29.625 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.83', 'mon.b': '172.21.15.94', 'mon.c': '172.21.15.111'} 2024-04-25T11:32:29.625 INFO:tasks.cephadm:First mon is mon.a on smithi083 2024-04-25T11:32:29.625 INFO:tasks.cephadm:First mgr is x 2024-04-25T11:32:29.625 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T11:32:29.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname $(hostname -s) 2024-04-25T11:32:29.649 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname $(hostname -s) 2024-04-25T11:32:29.678 DEBUG:teuthology.orchestra.run.smithi111:> sudo hostname $(hostname -s) 2024-04-25T11:32:29.701 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T11:32:29.701 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:32:29.837 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T11:32:29.973 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:32:29.974 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T11:32:29.975 DEBUG:teuthology.orchestra.run.smithi083:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T11:32:30.709 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 11:32 /home/ubuntu/cephtest/cephadm 2024-04-25T11:32:30.710 DEBUG:teuthology.orchestra.run.smithi094:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T11:32:31.307 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 11:32 /home/ubuntu/cephtest/cephadm 2024-04-25T11:32:31.307 DEBUG:teuthology.orchestra.run.smithi111:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T11:32:31.930 INFO:teuthology.orchestra.run.smithi111.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 11:32 /home/ubuntu/cephtest/cephadm 2024-04-25T11:32:31.931 DEBUG:teuthology.orchestra.run.smithi083:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T11:32:31.949 DEBUG:teuthology.orchestra.run.smithi094:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T11:32:31.968 DEBUG:teuthology.orchestra.run.smithi111:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T11:32:32.002 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T11:32:32.002 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T11:32:32.004 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T11:32:32.011 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T11:32:33.652 INFO:teuthology.orchestra.run.smithi094.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T11:32:33.661 INFO:teuthology.orchestra.run.smithi083.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T11:32:33.721 INFO:teuthology.orchestra.run.smithi111.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T11:33:20.801 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:33:20.801 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T11:33:20.801 INFO:teuthology.orchestra.run.smithi083.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T11:33:20.801 INFO:teuthology.orchestra.run.smithi083.stdout: "repo_digests": [ 2024-04-25T11:33:20.801 INFO:teuthology.orchestra.run.smithi083.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T11:33:20.801 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T11:33:20.802 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:33:26.057 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-25T11:33:26.057 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T11:33:26.057 INFO:teuthology.orchestra.run.smithi111.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T11:33:26.057 INFO:teuthology.orchestra.run.smithi111.stdout: "repo_digests": [ 2024-04-25T11:33:26.058 INFO:teuthology.orchestra.run.smithi111.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T11:33:26.058 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-04-25T11:33:26.058 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-25T11:33:36.338 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T11:33:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T11:33:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T11:33:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "repo_digests": [ 2024-04-25T11:33:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T11:33:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T11:33:36.339 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T11:33:36.363 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph 2024-04-25T11:33:36.395 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph 2024-04-25T11:33:36.422 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph 2024-04-25T11:33:36.454 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /etc/ceph 2024-04-25T11:33:36.481 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /etc/ceph 2024-04-25T11:33:36.505 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /etc/ceph 2024-04-25T11:33:36.532 INFO:tasks.cephadm:Writing seed config... 2024-04-25T11:33:36.534 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T11:33:36.534 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T11:33:36.534 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T11:33:36.534 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T11:33:36.534 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T11:33:36.534 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T11:33:36.534 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T11:33:36.534 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T11:33:36.534 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T11:33:36.534 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T11:33:36.534 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T11:33:36.535 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T11:33:36.535 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T11:33:36.535 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T11:33:36.535 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T11:33:36.535 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T11:33:36.535 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T11:33:36.535 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T11:33:36.535 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T11:33:36.535 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T11:33:36.535 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T11:33:36.535 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T11:33:36.535 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T11:33:36.535 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T11:33:36.536 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T11:33:36.536 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T11:33:36.536 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T11:33:36.536 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T11:33:36.536 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T11:33:36.536 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T11:33:36.536 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T11:33:36.536 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T11:33:36.536 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T11:33:36.536 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T11:33:36.536 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T11:33:36.536 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T11:33:36.536 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T11:33:36.536 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T11:33:36.537 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T11:33:36.537 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T11:33:36.537 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T11:33:36.537 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T11:33:36.537 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T11:33:36.537 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T11:33:36.537 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T11:33:36.537 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T11:33:36.537 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T11:33:36.537 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T11:33:36.538 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:33:36.538 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T11:33:36.554 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 7fcf54c6-02f7-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T11:33:36.555 DEBUG:teuthology.orchestra.run.smithi083:mon.a> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T11:33:36.597 DEBUG:teuthology.orchestra.run.smithi083:mgr.x> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T11:33:36.639 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T11:33:36.639 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.83 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:33:36.830 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:33:36.830 INFO:teuthology.orchestra.run.smithi083.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', '7fcf54c6-02f7-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.83', '--skip-admin-label'] 2024-04-25T11:33:36.830 INFO:teuthology.orchestra.run.smithi083.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T11:33:36.831 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying podman|docker is present... 2024-04-25T11:33:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T11:33:36.856 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying lvm2 is present... 2024-04-25T11:33:36.857 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying time synchronization is in place... 2024-04-25T11:33:36.864 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T11:33:36.864 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T11:33:36.870 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T11:33:36.870 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-25T11:33:36.876 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout enabled 2024-04-25T11:33:36.882 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout active 2024-04-25T11:33:36.882 INFO:teuthology.orchestra.run.smithi083.stdout:Unit chronyd.service is enabled and running 2024-04-25T11:33:36.882 INFO:teuthology.orchestra.run.smithi083.stdout:Repeating the final host check... 2024-04-25T11:33:36.920 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T11:33:36.920 INFO:teuthology.orchestra.run.smithi083.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T11:33:36.921 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl is present 2024-04-25T11:33:36.921 INFO:teuthology.orchestra.run.smithi083.stdout:lvcreate is present 2024-04-25T11:33:36.927 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T11:33:36.927 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T11:33:36.932 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T11:33:36.932 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-25T11:33:36.939 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout enabled 2024-04-25T11:33:36.945 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout active 2024-04-25T11:33:36.945 INFO:teuthology.orchestra.run.smithi083.stdout:Unit chronyd.service is enabled and running 2024-04-25T11:33:36.945 INFO:teuthology.orchestra.run.smithi083.stdout:Host looks OK 2024-04-25T11:33:36.945 INFO:teuthology.orchestra.run.smithi083.stdout:Cluster fsid: 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:33:36.945 INFO:teuthology.orchestra.run.smithi083.stdout:Acquiring lock 140429496597664 on /run/cephadm/7fcf54c6-02f7-11ef-bc93-c7b262605968.lock 2024-04-25T11:33:36.945 INFO:teuthology.orchestra.run.smithi083.stdout:Lock 140429496597664 acquired on /run/cephadm/7fcf54c6-02f7-11ef-bc93-c7b262605968.lock 2024-04-25T11:33:36.946 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying IP 172.21.15.83 port 3300 ... 2024-04-25T11:33:36.946 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying IP 172.21.15.83 port 6789 ... 2024-04-25T11:33:36.946 INFO:teuthology.orchestra.run.smithi083.stdout:Base mon IP(s) is [172.21.15.83:3300, 172.21.15.83:6789], mon addrv is [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-04-25T11:33:36.949 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.83 metric 100 2024-04-25T11:33:36.949 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.83 metric 100 2024-04-25T11:33:36.952 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T11:33:36.952 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-25T11:33:36.952 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-25T11:33:36.954 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T11:33:36.954 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T11:33:36.954 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T11:33:36.954 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T11:33:36.954 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout inet6 fe80::ea47:558e:effa:6a90/64 scope link noprefixroute 2024-04-25T11:33:36.955 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T11:33:36.955 INFO:teuthology.orchestra.run.smithi083.stdout:Mon IP `172.21.15.83` is in CIDR network `172.21.0.0/20` 2024-04-25T11:33:36.955 INFO:teuthology.orchestra.run.smithi083.stdout:Mon IP `172.21.15.83` is in CIDR network `172.21.0.0/20` 2024-04-25T11:33:36.955 INFO:teuthology.orchestra.run.smithi083.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T11:33:36.956 INFO:teuthology.orchestra.run.smithi083.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T11:33:36.956 INFO:teuthology.orchestra.run.smithi083.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T11:33:37.484 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T11:33:37.485 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T11:33:37.485 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T11:33:37.485 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T11:33:37.485 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T11:33:37.485 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T11:33:37.485 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T11:33:38.406 INFO:teuthology.orchestra.run.smithi083.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T11:33:38.406 INFO:teuthology.orchestra.run.smithi083.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T11:33:38.406 INFO:teuthology.orchestra.run.smithi083.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T11:33:39.395 INFO:teuthology.orchestra.run.smithi083.stdout:stat: stdout 167 167 2024-04-25T11:33:39.395 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial keys... 2024-04-25T11:33:40.239 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQCTPypm2mptKxAApa1HD1EKd8gwmF+P1Zencw== 2024-04-25T11:33:41.188 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQCUPypmJH1/IhAAUVC/H7yqHORYqjIEZUUFTQ== 2024-04-25T11:33:42.106 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQCVPypmxVZvIBAAKC5/3JVwQMe7xXCI2xQgDA== 2024-04-25T11:33:42.107 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial monmap... 2024-04-25T11:33:46.071 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T11:33:46.071 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T11:33:46.071 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:33:46.071 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T11:33:46.071 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool for a [v2:172.21.15.83:3300,v1:172.21.15.83:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T11:33:46.071 INFO:teuthology.orchestra.run.smithi083.stdout:setting min_mon_release = quincy 2024-04-25T11:33:46.071 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: set fsid to 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:33:46.071 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T11:33:46.071 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:33:46.071 INFO:teuthology.orchestra.run.smithi083.stdout:Creating mon... 2024-04-25T11:33:54.993 INFO:teuthology.orchestra.run.smithi083.stdout:create mon.a on 2024-04-25T11:33:55.207 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T11:33:55.358 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T11:33:55.539 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968.target. 2024-04-25T11:33:55.540 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968.target. 2024-04-25T11:33:55.774 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.a 2024-04-25T11:33:55.774 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T11:33:55.936 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968.target.wants/ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@.service. 2024-04-25T11:33:56.441 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T11:33:56.441 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-04-25T11:33:56.449 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T11:33:56.449 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-25T11:33:56.449 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-04-25T11:33:56.449 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T11:33:56.449 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mon to start... 2024-04-25T11:33:56.449 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mon... 2024-04-25T11:33:57.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T11:33:57.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout id: 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:33:57.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T11:33:57.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-25T11:33:57.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout services: 2024-04-25T11:33:57.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.588575s) 2024-04-25T11:33:57.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T11:33:57.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T11:33:57.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-25T11:33:57.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout data: 2024-04-25T11:33:57.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T11:33:57.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T11:33:57.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T11:33:57.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T11:33:57.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-25T11:33:57.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.916+0000 7fe61df4a700 1 Processor -- start 2024-04-25T11:33:57.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.916+0000 7fe61df4a700 1 -- start start 2024-04-25T11:33:57.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.916+0000 7fe61df4a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618106c60 0x7fe618107080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:57.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.916+0000 7fe61df4a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe618107650 con 0x7fe618106c60 2024-04-25T11:33:57.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.916+0000 7fe6177fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618106c60 0x7fe618107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:57.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.916+0000 7fe6177fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618106c60 0x7fe618107080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56596/0 (socket says 172.21.15.83:56596) 2024-04-25T11:33:57.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.916+0000 7fe6177fe700 1 -- 172.21.15.83:0/3382663850 learned_addr learned my addr 172.21.15.83:0/3382663850 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:33:57.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.916+0000 7fe6177fe700 1 -- 172.21.15.83:0/3382663850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe618107e60 con 0x7fe618106c60 2024-04-25T11:33:57.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.917+0000 7fe6177fe700 1 --2- 172.21.15.83:0/3382663850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618106c60 0x7fe618107080 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe6080098a0 tx=0x7fe608009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263603fc2fa7c556 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:57.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.917+0000 7fe6167fc700 1 -- 172.21.15.83:0/3382663850 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe608004160 con 0x7fe618106c60 2024-04-25T11:33:57.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.917+0000 7fe6167fc700 1 -- 172.21.15.83:0/3382663850 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe608004300 con 0x7fe618106c60 2024-04-25T11:33:57.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.917+0000 7fe6167fc700 1 -- 172.21.15.83:0/3382663850 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6080036a0 con 0x7fe618106c60 2024-04-25T11:33:57.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.917+0000 7fe61df4a700 1 -- 172.21.15.83:0/3382663850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618106c60 msgr2=0x7fe618107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:57.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.917+0000 7fe61df4a700 1 --2- 172.21.15.83:0/3382663850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618106c60 0x7fe618107080 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe6080098a0 tx=0x7fe608009bb0 comp rx=0 tx=0).stop 2024-04-25T11:33:57.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.917+0000 7fe61df4a700 1 -- 172.21.15.83:0/3382663850 shutdown_connections 2024-04-25T11:33:57.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.917+0000 7fe61df4a700 1 --2- 172.21.15.83:0/3382663850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618106c60 0x7fe618107080 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:57.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.917+0000 7fe61df4a700 1 -- 172.21.15.83:0/3382663850 >> 172.21.15.83:0/3382663850 conn(0x7fe618102200 msgr2=0x7fe618104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:57.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.917+0000 7fe61df4a700 1 -- 172.21.15.83:0/3382663850 shutdown_connections 2024-04-25T11:33:57.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.917+0000 7fe61df4a700 1 -- 172.21.15.83:0/3382663850 wait complete. 2024-04-25T11:33:57.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.918+0000 7fe61df4a700 1 Processor -- start 2024-04-25T11:33:57.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.918+0000 7fe61df4a700 1 -- start start 2024-04-25T11:33:57.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.918+0000 7fe61df4a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618106c60 0x7fe61819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:57.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.918+0000 7fe61df4a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe618108390 con 0x7fe618106c60 2024-04-25T11:33:57.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.918+0000 7fe6177fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618106c60 0x7fe61819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:57.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.918+0000 7fe6177fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618106c60 0x7fe61819bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56604/0 (socket says 172.21.15.83:56604) 2024-04-25T11:33:57.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.918+0000 7fe6177fe700 1 -- 172.21.15.83:0/2006257771 learned_addr learned my addr 172.21.15.83:0/2006257771 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:33:57.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.919+0000 7fe6177fe700 1 -- 172.21.15.83:0/2006257771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61819c1d0 con 0x7fe618106c60 2024-04-25T11:33:57.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.919+0000 7fe6177fe700 1 --2- 172.21.15.83:0/2006257771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618106c60 0x7fe61819bc90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe608000c00 tx=0x7fe608004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:57.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.919+0000 7fe614ff9700 1 -- 172.21.15.83:0/2006257771 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe608003db0 con 0x7fe618106c60 2024-04-25T11:33:57.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.919+0000 7fe614ff9700 1 -- 172.21.15.83:0/2006257771 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe608011630 con 0x7fe618106c60 2024-04-25T11:33:57.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.919+0000 7fe61df4a700 1 -- 172.21.15.83:0/2006257771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61819c460 con 0x7fe618106c60 2024-04-25T11:33:57.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.919+0000 7fe614ff9700 1 -- 172.21.15.83:0/2006257771 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe608011950 con 0x7fe618106c60 2024-04-25T11:33:57.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.919+0000 7fe61df4a700 1 -- 172.21.15.83:0/2006257771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61819f150 con 0x7fe618106c60 2024-04-25T11:33:57.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.919+0000 7fe614ff9700 1 -- 172.21.15.83:0/2006257771 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe608011af0 con 0x7fe618106c60 2024-04-25T11:33:57.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.919+0000 7fe614ff9700 1 -- 172.21.15.83:0/2006257771 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe60801c450 con 0x7fe618106c60 2024-04-25T11:33:57.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.920+0000 7fe61df4a700 1 -- 172.21.15.83:0/2006257771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6180640a0 con 0x7fe618106c60 2024-04-25T11:33:57.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.922+0000 7fe614ff9700 1 -- 172.21.15.83:0/2006257771 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe608004260 con 0x7fe618106c60 2024-04-25T11:33:57.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.991+0000 7fe61df4a700 1 -- 172.21.15.83:0/2006257771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fe61802e090 con 0x7fe618106c60 2024-04-25T11:33:57.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.992+0000 7fe614ff9700 1 -- 172.21.15.83:0/2006257771 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fe60801b450 con 0x7fe618106c60 2024-04-25T11:33:57.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.992+0000 7fe61df4a700 1 -- 172.21.15.83:0/2006257771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618106c60 msgr2=0x7fe61819bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:57.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.993+0000 7fe61df4a700 1 --2- 172.21.15.83:0/2006257771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618106c60 0x7fe61819bc90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe608000c00 tx=0x7fe608004750 comp rx=0 tx=0).stop 2024-04-25T11:33:57.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.993+0000 7fe61df4a700 1 -- 172.21.15.83:0/2006257771 shutdown_connections 2024-04-25T11:33:57.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.993+0000 7fe61df4a700 1 --2- 172.21.15.83:0/2006257771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618106c60 0x7fe61819bc90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:57.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.993+0000 7fe61df4a700 1 -- 172.21.15.83:0/2006257771 >> 172.21.15.83:0/2006257771 conn(0x7fe618102200 msgr2=0x7fe618062f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:57.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.993+0000 7fe61df4a700 1 -- 172.21.15.83:0/2006257771 shutdown_connections 2024-04-25T11:33:57.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:56.993+0000 7fe61df4a700 1 -- 172.21.15.83:0/2006257771 wait complete. 2024-04-25T11:33:57.535 INFO:teuthology.orchestra.run.smithi083.stdout:mon is available 2024-04-25T11:33:57.535 INFO:teuthology.orchestra.run.smithi083.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T11:33:58.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:33:58 smithi083 ceph-mon[28952]: from='client.? 172.21.15.83:0/2289835922' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T11:33:58.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-25T11:33:58.546 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T11:33:58.546 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout fsid = 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:33:58.546 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T11:33:58.546 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-04-25T11:33:58.546 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T11:33:58.546 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T11:33:58.546 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T11:33:58.546 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T11:33:58.546 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-25T11:33:58.546 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T11:33:58.546 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T11:33:58.546 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-25T11:33:58.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T11:33:58.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T11:33:58.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T11:33:58.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T11:33:58.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T11:33:58.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T11:33:58.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T11:33:58.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.967+0000 7f3bfe669700 1 Processor -- start 2024-04-25T11:33:58.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.967+0000 7f3bfe669700 1 -- start start 2024-04-25T11:33:58.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.967+0000 7f3bfe669700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf807de50 0x7f3bf807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:58.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.967+0000 7f3bfe669700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf807c8d0 con 0x7f3bf807de50 2024-04-25T11:33:58.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.967+0000 7f3bf7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf807de50 0x7f3bf807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:58.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.967+0000 7f3bf7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf807de50 0x7f3bf807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56618/0 (socket says 172.21.15.83:56618) 2024-04-25T11:33:58.547 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.967+0000 7f3bf7fff700 1 -- 172.21.15.83:0/3737251335 learned_addr learned my addr 172.21.15.83:0/3737251335 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:33:58.548 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.967+0000 7f3bf7fff700 1 -- 172.21.15.83:0/3737251335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf807ca50 con 0x7f3bf807de50 2024-04-25T11:33:58.548 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.968+0000 7f3bf7fff700 1 --2- 172.21.15.83:0/3737251335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf807de50 0x7f3bf807c300 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3be80098a0 tx=0x7f3be8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1215a3654cde800b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:58.548 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.968+0000 7f3bf6ffd700 1 -- 172.21.15.83:0/3737251335 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be8004160 con 0x7f3bf807de50 2024-04-25T11:33:58.548 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.968+0000 7f3bf6ffd700 1 -- 172.21.15.83:0/3737251335 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be8004300 con 0x7f3bf807de50 2024-04-25T11:33:58.548 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.968+0000 7f3bf6ffd700 1 -- 172.21.15.83:0/3737251335 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be80036a0 con 0x7f3bf807de50 2024-04-25T11:33:58.548 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.968+0000 7f3bfe669700 1 -- 172.21.15.83:0/3737251335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf807de50 msgr2=0x7f3bf807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:58.548 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.968+0000 7f3bfe669700 1 --2- 172.21.15.83:0/3737251335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf807de50 0x7f3bf807c300 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3be80098a0 tx=0x7f3be8009bb0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.548 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.968+0000 7f3bfe669700 1 -- 172.21.15.83:0/3737251335 shutdown_connections 2024-04-25T11:33:58.548 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.968+0000 7f3bfe669700 1 --2- 172.21.15.83:0/3737251335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf807de50 0x7f3bf807c300 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.548 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.968+0000 7f3bfe669700 1 -- 172.21.15.83:0/3737251335 >> 172.21.15.83:0/3737251335 conn(0x7f3bf8103bc0 msgr2=0x7f3bf8105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:58.548 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.968+0000 7f3bfe669700 1 -- 172.21.15.83:0/3737251335 shutdown_connections 2024-04-25T11:33:58.548 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.968+0000 7f3bfe669700 1 -- 172.21.15.83:0/3737251335 wait complete. 2024-04-25T11:33:58.548 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.969+0000 7f3bfe669700 1 Processor -- start 2024-04-25T11:33:58.548 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.969+0000 7f3bfe669700 1 -- start start 2024-04-25T11:33:58.549 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.969+0000 7f3bfe669700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf807de50 0x7f3bf819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:58.549 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.969+0000 7f3bfe669700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf807cf20 con 0x7f3bf807de50 2024-04-25T11:33:58.549 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.969+0000 7f3bf7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf807de50 0x7f3bf819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:58.549 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.969+0000 7f3bf7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf807de50 0x7f3bf819dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56632/0 (socket says 172.21.15.83:56632) 2024-04-25T11:33:58.549 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.969+0000 7f3bf7fff700 1 -- 172.21.15.83:0/2289835922 learned_addr learned my addr 172.21.15.83:0/2289835922 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:33:58.549 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.970+0000 7f3bf7fff700 1 -- 172.21.15.83:0/2289835922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf819e320 con 0x7f3bf807de50 2024-04-25T11:33:58.549 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.970+0000 7f3bf7fff700 1 --2- 172.21.15.83:0/2289835922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf807de50 0x7f3bf819dde0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3be8000c00 tx=0x7f3be8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:58.549 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.970+0000 7f3bf57fa700 1 -- 172.21.15.83:0/2289835922 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be8003db0 con 0x7f3bf807de50 2024-04-25T11:33:58.549 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.970+0000 7f3bf57fa700 1 -- 172.21.15.83:0/2289835922 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be8011630 con 0x7f3bf807de50 2024-04-25T11:33:58.549 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.970+0000 7f3bfe669700 1 -- 172.21.15.83:0/2289835922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf819e5b0 con 0x7f3bf807de50 2024-04-25T11:33:58.549 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.970+0000 7f3bf57fa700 1 -- 172.21.15.83:0/2289835922 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3be8011950 con 0x7f3bf807de50 2024-04-25T11:33:58.549 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.970+0000 7f3bfe669700 1 -- 172.21.15.83:0/2289835922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf81a12a0 con 0x7f3bf807de50 2024-04-25T11:33:58.549 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.970+0000 7f3bf57fa700 1 -- 172.21.15.83:0/2289835922 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3be8011af0 con 0x7f3bf807de50 2024-04-25T11:33:58.549 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.970+0000 7f3bf57fa700 1 -- 172.21.15.83:0/2289835922 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3be801c450 con 0x7f3bf807de50 2024-04-25T11:33:58.550 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.971+0000 7f3bfe669700 1 -- 172.21.15.83:0/2289835922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf80640a0 con 0x7f3bf807de50 2024-04-25T11:33:58.550 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:57.973+0000 7f3bf57fa700 1 -- 172.21.15.83:0/2289835922 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3be8032020 con 0x7f3bf807de50 2024-04-25T11:33:58.550 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.041+0000 7f3bfe669700 1 -- 172.21.15.83:0/2289835922 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f3bf81a2860 con 0x7f3bf807de50 2024-04-25T11:33:58.550 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.044+0000 7f3bf57fa700 1 -- 172.21.15.83:0/2289835922 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f3be8004260 con 0x7f3bf807de50 2024-04-25T11:33:58.550 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.044+0000 7f3bf57fa700 1 -- 172.21.15.83:0/2289835922 <== mon.0 v2:172.21.15.83:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3be801c650 con 0x7f3bf807de50 2024-04-25T11:33:58.550 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.044+0000 7f3bfe669700 1 -- 172.21.15.83:0/2289835922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf807de50 msgr2=0x7f3bf819dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:58.550 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.044+0000 7f3bfe669700 1 --2- 172.21.15.83:0/2289835922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf807de50 0x7f3bf819dde0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3be8000c00 tx=0x7f3be8004750 comp rx=0 tx=0).stop 2024-04-25T11:33:58.550 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.045+0000 7f3bfe669700 1 -- 172.21.15.83:0/2289835922 shutdown_connections 2024-04-25T11:33:58.550 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.045+0000 7f3bfe669700 1 --2- 172.21.15.83:0/2289835922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bf807de50 0x7f3bf819dde0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:58.550 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.045+0000 7f3bfe669700 1 -- 172.21.15.83:0/2289835922 >> 172.21.15.83:0/2289835922 conn(0x7f3bf8103bc0 msgr2=0x7f3bf81049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:58.550 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.045+0000 7f3bfe669700 1 -- 172.21.15.83:0/2289835922 shutdown_connections 2024-04-25T11:33:58.550 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.045+0000 7f3bfe669700 1 -- 172.21.15.83:0/2289835922 wait complete. 2024-04-25T11:33:58.550 INFO:teuthology.orchestra.run.smithi083.stdout:Generating new minimal ceph.conf... 2024-04-25T11:33:59.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.966+0000 7ff6456e7700 1 Processor -- start 2024-04-25T11:33:59.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.966+0000 7ff6456e7700 1 -- start start 2024-04-25T11:33:59.618 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.967+0000 7ff6456e7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff640108600 0x7ff640108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:59.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.967+0000 7ff6456e7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff640108ff0 con 0x7ff640108600 2024-04-25T11:33:59.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.967+0000 7ff63effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff640108600 0x7ff640108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:59.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.967+0000 7ff63effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff640108600 0x7ff640108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56648/0 (socket says 172.21.15.83:56648) 2024-04-25T11:33:59.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.967+0000 7ff63effd700 1 -- 172.21.15.83:0/4097295991 learned_addr learned my addr 172.21.15.83:0/4097295991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:33:59.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.967+0000 7ff63effd700 1 -- 172.21.15.83:0/4097295991 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6401097b0 con 0x7ff640108600 2024-04-25T11:33:59.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.967+0000 7ff63effd700 1 --2- 172.21.15.83:0/4097295991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff640108600 0x7ff640108a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff6300098a0 tx=0x7ff630009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c5e6c5524d319 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:59.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.967+0000 7ff63dffb700 1 -- 172.21.15.83:0/4097295991 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff630004160 con 0x7ff640108600 2024-04-25T11:33:59.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.967+0000 7ff63dffb700 1 -- 172.21.15.83:0/4097295991 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff630004300 con 0x7ff640108600 2024-04-25T11:33:59.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.967+0000 7ff63dffb700 1 -- 172.21.15.83:0/4097295991 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff630011af0 con 0x7ff640108600 2024-04-25T11:33:59.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.968+0000 7ff6456e7700 1 -- 172.21.15.83:0/4097295991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff640108600 msgr2=0x7ff640108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:59.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.968+0000 7ff6456e7700 1 --2- 172.21.15.83:0/4097295991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff640108600 0x7ff640108a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff6300098a0 tx=0x7ff630009bb0 comp rx=0 tx=0).stop 2024-04-25T11:33:59.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.968+0000 7ff6456e7700 1 -- 172.21.15.83:0/4097295991 shutdown_connections 2024-04-25T11:33:59.619 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.968+0000 7ff6456e7700 1 --2- 172.21.15.83:0/4097295991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff640108600 0x7ff640108a20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:59.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.968+0000 7ff6456e7700 1 -- 172.21.15.83:0/4097295991 >> 172.21.15.83:0/4097295991 conn(0x7ff640103bc0 msgr2=0x7ff640105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:59.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.968+0000 7ff6456e7700 1 -- 172.21.15.83:0/4097295991 shutdown_connections 2024-04-25T11:33:59.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.968+0000 7ff6456e7700 1 -- 172.21.15.83:0/4097295991 wait complete. 2024-04-25T11:33:59.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.969+0000 7ff6456e7700 1 Processor -- start 2024-04-25T11:33:59.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.969+0000 7ff6456e7700 1 -- start start 2024-04-25T11:33:59.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.969+0000 7ff6456e7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff640108600 0x7ff64007db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:33:59.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.969+0000 7ff6456e7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff64010c9e0 con 0x7ff640108600 2024-04-25T11:33:59.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.969+0000 7ff63effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff640108600 0x7ff64007db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:33:59.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.969+0000 7ff63effd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff640108600 0x7ff64007db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56660/0 (socket says 172.21.15.83:56660) 2024-04-25T11:33:59.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.969+0000 7ff63effd700 1 -- 172.21.15.83:0/2354317298 learned_addr learned my addr 172.21.15.83:0/2354317298 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:33:59.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.969+0000 7ff63effd700 1 -- 172.21.15.83:0/2354317298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64007e090 con 0x7ff640108600 2024-04-25T11:33:59.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.970+0000 7ff63effd700 1 --2- 172.21.15.83:0/2354317298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff640108600 0x7ff64007db50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff630009fb0 tx=0x7ff630011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:33:59.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.970+0000 7ff627fff700 1 -- 172.21.15.83:0/2354317298 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6300036a0 con 0x7ff640108600 2024-04-25T11:33:59.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.970+0000 7ff627fff700 1 -- 172.21.15.83:0/2354317298 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff63001a430 con 0x7ff640108600 2024-04-25T11:33:59.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.970+0000 7ff6456e7700 1 -- 172.21.15.83:0/2354317298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64007c280 con 0x7ff640108600 2024-04-25T11:33:59.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.970+0000 7ff627fff700 1 -- 172.21.15.83:0/2354317298 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff630019550 con 0x7ff640108600 2024-04-25T11:33:59.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.970+0000 7ff6456e7700 1 -- 172.21.15.83:0/2354317298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64007c7c0 con 0x7ff640108600 2024-04-25T11:33:59.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.970+0000 7ff627fff700 1 -- 172.21.15.83:0/2354317298 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff630019b90 con 0x7ff640108600 2024-04-25T11:33:59.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.970+0000 7ff627fff700 1 -- 172.21.15.83:0/2354317298 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff630024730 con 0x7ff640108600 2024-04-25T11:33:59.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.971+0000 7ff6456e7700 1 -- 172.21.15.83:0/2354317298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff64010cb60 con 0x7ff640108600 2024-04-25T11:33:59.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:58.973+0000 7ff627fff700 1 -- 172.21.15.83:0/2354317298 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff630043e50 con 0x7ff640108600 2024-04-25T11:33:59.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:59.041+0000 7ff6456e7700 1 -- 172.21.15.83:0/2354317298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff64007ceb0 con 0x7ff640108600 2024-04-25T11:33:59.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:59.041+0000 7ff627fff700 1 -- 172.21.15.83:0/2354317298 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7ff630032020 con 0x7ff640108600 2024-04-25T11:33:59.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:59.042+0000 7ff6456e7700 1 -- 172.21.15.83:0/2354317298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff640108600 msgr2=0x7ff64007db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:33:59.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:59.042+0000 7ff6456e7700 1 --2- 172.21.15.83:0/2354317298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff640108600 0x7ff64007db50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff630009fb0 tx=0x7ff630011940 comp rx=0 tx=0).stop 2024-04-25T11:33:59.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:59.042+0000 7ff6456e7700 1 -- 172.21.15.83:0/2354317298 shutdown_connections 2024-04-25T11:33:59.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:59.042+0000 7ff6456e7700 1 --2- 172.21.15.83:0/2354317298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff640108600 0x7ff64007db50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:33:59.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:59.042+0000 7ff6456e7700 1 -- 172.21.15.83:0/2354317298 >> 172.21.15.83:0/2354317298 conn(0x7ff640103bc0 msgr2=0x7ff640104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:33:59.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:59.042+0000 7ff6456e7700 1 -- 172.21.15.83:0/2354317298 shutdown_connections 2024-04-25T11:33:59.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:33:59.042+0000 7ff6456e7700 1 -- 172.21.15.83:0/2354317298 wait complete. 2024-04-25T11:33:59.622 INFO:teuthology.orchestra.run.smithi083.stdout:Restarting the monitor... 2024-04-25T11:33:59.644 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:33:59 smithi083 ceph-mon[28952]: from='client.? 172.21.15.83:0/2354317298' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:33:59.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:33:59 smithi083 systemd[1]: Stopping Ceph mon.a for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:33:59.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:33:59 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a[28948]: 2024-04-25T11:33:59.901+0000 7f4e43e3a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:33:59.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:33:59 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a[28948]: 2024-04-25T11:33:59.901+0000 7f4e43e3a700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T11:34:00.387 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:00 smithi083 podman[29199]: 2024-04-25 11:34:00.09601139 +0000 UTC m=+0.293596166 container died bb37c080b86847886ef91c895f59f3a0af0b99f548f4aa885a122c067020c0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:34:00.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:00 smithi083 podman[29199]: 2024-04-25 11:34:00.593148015 +0000 UTC m=+0.790732832 container cleanup bb37c080b86847886ef91c895f59f3a0af0b99f548f4aa885a122c067020c0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T11:34:00.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:00 smithi083 bash[29199]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a 2024-04-25T11:34:01.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:00 smithi083 podman[29214]: 2024-04-25 11:34:00.91920386 +0000 UTC m=+0.816140543 container remove bb37c080b86847886ef91c895f59f3a0af0b99f548f4aa885a122c067020c0ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T11:34:01.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 systemd[1]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T11:34:01.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 systemd[1]: Stopped Ceph mon.a for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:34:01.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 systemd[1]: Starting Ceph mon.a for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:34:01.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 podman[29311]: 2024-04-25 11:34:01.261656971 +0000 UTC m=+0.096614868 container create b35e79aacc4caa453c37b1077e06f233a5a06ac3bdda6cd43828836179f78708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:34:01.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 podman[29311]: 2024-04-25 11:34:01.182313604 +0000 UTC m=+0.017271501 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T11:34:01.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 podman[29311]: 2024-04-25 11:34:01.455689036 +0000 UTC m=+0.290646935 container init b35e79aacc4caa453c37b1077e06f233a5a06ac3bdda6cd43828836179f78708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T11:34:01.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 podman[29311]: 2024-04-25 11:34:01.459273146 +0000 UTC m=+0.294231046 container start b35e79aacc4caa453c37b1077e06f233a5a06ac3bdda6cd43828836179f78708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T11:34:01.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T11:34:01.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T11:34:01.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: pidfile_write: ignore empty --pid-file 2024-04-25T11:34:01.519 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: load: jerasure load: lrc 2024-04-25T11:34:01.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: RocksDB version: 7.9.2 2024-04-25T11:34:01.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Git sha 0 2024-04-25T11:34:01.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T11:34:01.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: DB SUMMARY 2024-04-25T11:34:01.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: DB Session ID: CDXZ0B4A7UUZM9JDSI25 2024-04-25T11:34:01.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: CURRENT file: CURRENT 2024-04-25T11:34:01.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T11:34:01.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T11:34:01.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T11:34:01.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104884 ; 2024-04-25T11:34:01.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.error_if_exists: 0 2024-04-25T11:34:01.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.create_if_missing: 0 2024-04-25T11:34:01.520 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.paranoid_checks: 1 2024-04-25T11:34:01.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T11:34:01.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T11:34:01.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T11:34:01.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.env: 0x5600f90accc0 2024-04-25T11:34:01.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T11:34:01.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.info_log: 0x5600fb3a5be0 2024-04-25T11:34:01.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T11:34:01.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.statistics: (nil) 2024-04-25T11:34:01.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.use_fsync: 0 2024-04-25T11:34:01.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_log_file_size: 0 2024-04-25T11:34:01.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T11:34:01.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T11:34:01.521 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T11:34:01.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T11:34:01.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.allow_fallocate: 1 2024-04-25T11:34:01.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T11:34:01.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T11:34:01.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.use_direct_reads: 0 2024-04-25T11:34:01.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T11:34:01.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T11:34:01.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.db_log_dir: 2024-04-25T11:34:01.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.wal_dir: 2024-04-25T11:34:01.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T11:34:01.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T11:34:01.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T11:34:01.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T11:34:01.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T11:34:01.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T11:34:01.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T11:34:01.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T11:34:01.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.write_buffer_manager: 0x5600fb3390e0 2024-04-25T11:34:01.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T11:34:01.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T11:34:01.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T11:34:01.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T11:34:01.523 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T11:34:01.524 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T11:34:01.524 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T11:34:01.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T11:34:01.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.unordered_write: 0 2024-04-25T11:34:01.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T11:34:01.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T11:34:01.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T11:34:01.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T11:34:01.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.row_cache: None 2024-04-25T11:34:01.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.wal_filter: None 2024-04-25T11:34:01.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T11:34:01.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T11:34:01.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.two_write_queues: 0 2024-04-25T11:34:01.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T11:34:01.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.wal_compression: 0 2024-04-25T11:34:01.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.atomic_flush: 0 2024-04-25T11:34:01.525 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T11:34:01.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T11:34:01.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T11:34:01.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.log_readahead_size: 0 2024-04-25T11:34:01.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T11:34:01.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T11:34:01.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T11:34:01.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T11:34:01.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T11:34:01.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T11:34:01.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T11:34:01.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_background_jobs: 2 2024-04-25T11:34:01.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_background_compactions: -1 2024-04-25T11:34:01.526 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_subcompactions: 1 2024-04-25T11:34:01.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T11:34:01.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T11:34:01.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T11:34:01.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T11:34:01.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T11:34:01.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T11:34:01.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T11:34:01.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T11:34:01.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_open_files: -1 2024-04-25T11:34:01.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T11:34:01.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T11:34:01.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T11:34:01.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T11:34:01.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_background_flushes: -1 2024-04-25T11:34:01.527 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Compression algorithms supported: 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: kZSTD supported: 0 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: kXpressCompression supported: 0 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: kZlibCompression supported: 1 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: kSnappyCompression supported: 1 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: kLZ4Compression supported: 1 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: kBZip2Compression supported: 0 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.merge_operator: 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compaction_filter: None 2024-04-25T11:34:01.528 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T11:34:01.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T11:34:01.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T11:34:01.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T11:34:01.529 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5600fb3a5dc0) 2024-04-25T11:34:01.529 INFO:journalctl@ceph.mon.a.smithi083.stdout: cache_index_and_filter_blocks: 1 2024-04-25T11:34:01.529 INFO:journalctl@ceph.mon.a.smithi083.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T11:34:01.529 INFO:journalctl@ceph.mon.a.smithi083.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T11:34:01.529 INFO:journalctl@ceph.mon.a.smithi083.stdout: pin_top_level_index_and_filter: 1 2024-04-25T11:34:01.529 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_type: 0 2024-04-25T11:34:01.529 INFO:journalctl@ceph.mon.a.smithi083.stdout: data_block_index_type: 0 2024-04-25T11:34:01.529 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_shortening: 1 2024-04-25T11:34:01.529 INFO:journalctl@ceph.mon.a.smithi083.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T11:34:01.529 INFO:journalctl@ceph.mon.a.smithi083.stdout: checksum: 4 2024-04-25T11:34:01.529 INFO:journalctl@ceph.mon.a.smithi083.stdout: no_block_cache: 0 2024-04-25T11:34:01.530 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache: 0x5600fb36b1f0 2024-04-25T11:34:01.530 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_name: BinnedLRUCache 2024-04-25T11:34:01.530 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_options: 2024-04-25T11:34:01.530 INFO:journalctl@ceph.mon.a.smithi083.stdout: capacity : 536870912 2024-04-25T11:34:01.530 INFO:journalctl@ceph.mon.a.smithi083.stdout: num_shard_bits : 4 2024-04-25T11:34:01.530 INFO:journalctl@ceph.mon.a.smithi083.stdout: strict_capacity_limit : 0 2024-04-25T11:34:01.530 INFO:journalctl@ceph.mon.a.smithi083.stdout: high_pri_pool_ratio: 0.000 2024-04-25T11:34:01.530 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_compressed: (nil) 2024-04-25T11:34:01.530 INFO:journalctl@ceph.mon.a.smithi083.stdout: persistent_cache: (nil) 2024-04-25T11:34:01.530 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_size: 4096 2024-04-25T11:34:01.530 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_size_deviation: 10 2024-04-25T11:34:01.530 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_restart_interval: 16 2024-04-25T11:34:01.530 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_block_restart_interval: 1 2024-04-25T11:34:01.530 INFO:journalctl@ceph.mon.a.smithi083.stdout: metadata_block_size: 4096 2024-04-25T11:34:01.531 INFO:journalctl@ceph.mon.a.smithi083.stdout: partition_filters: 0 2024-04-25T11:34:01.531 INFO:journalctl@ceph.mon.a.smithi083.stdout: use_delta_encoding: 1 2024-04-25T11:34:01.531 INFO:journalctl@ceph.mon.a.smithi083.stdout: filter_policy: bloomfilter 2024-04-25T11:34:01.531 INFO:journalctl@ceph.mon.a.smithi083.stdout: whole_key_filtering: 1 2024-04-25T11:34:01.531 INFO:journalctl@ceph.mon.a.smithi083.stdout: verify_compression: 0 2024-04-25T11:34:01.531 INFO:journalctl@ceph.mon.a.smithi083.stdout: read_amp_bytes_per_bit: 0 2024-04-25T11:34:01.531 INFO:journalctl@ceph.mon.a.smithi083.stdout: format_version: 5 2024-04-25T11:34:01.531 INFO:journalctl@ceph.mon.a.smithi083.stdout: enable_index_compression: 1 2024-04-25T11:34:01.531 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_align: 0 2024-04-25T11:34:01.531 INFO:journalctl@ceph.mon.a.smithi083.stdout: max_auto_readahead_size: 262144 2024-04-25T11:34:01.531 INFO:journalctl@ceph.mon.a.smithi083.stdout: prepopulate_block_cache: 0 2024-04-25T11:34:01.531 INFO:journalctl@ceph.mon.a.smithi083.stdout: initial_auto_readahead_size: 8192 2024-04-25T11:34:01.531 INFO:journalctl@ceph.mon.a.smithi083.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T11:34:01.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T11:34:01.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T11:34:01.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compression: NoCompression 2024-04-25T11:34:01.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T11:34:01.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T11:34:01.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T11:34:01.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.num_levels: 7 2024-04-25T11:34:01.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T11:34:01.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T11:34:01.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T11:34:01.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T11:34:01.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T11:34:01.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T11:34:01.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T11:34:01.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T11:34:01.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T11:34:01.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T11:34:01.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T11:34:01.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T11:34:01.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T11:34:01.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T11:34:01.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T11:34:01.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T11:34:01.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T11:34:01.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T11:34:01.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T11:34:01.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T11:34:01.533 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T11:34:01.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T11:34:01.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T11:34:01.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T11:34:01.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T11:34:01.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T11:34:01.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T11:34:01.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T11:34:01.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T11:34:01.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T11:34:01.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T11:34:01.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T11:34:01.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T11:34:01.534 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T11:34:01.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T11:34:01.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T11:34:01.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T11:34:01.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T11:34:01.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T11:34:01.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T11:34:01.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T11:34:01.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T11:34:01.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T11:34:01.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T11:34:01.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T11:34:01.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T11:34:01.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T11:34:01.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T11:34:01.535 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T11:34:01.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T11:34:01.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T11:34:01.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T11:34:01.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T11:34:01.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T11:34:01.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.inplace_update_support: 0 2024-04-25T11:34:01.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T11:34:01.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T11:34:01.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T11:34:01.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T11:34:01.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.bloom_locality: 0 2024-04-25T11:34:01.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.max_successive_merges: 0 2024-04-25T11:34:01.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T11:34:01.536 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T11:34:01.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T11:34:01.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T11:34:01.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.ttl: 2592000 2024-04-25T11:34:01.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T11:34:01.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T11:34:01.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T11:34:01.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.enable_blob_files: false 2024-04-25T11:34:01.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.min_blob_size: 0 2024-04-25T11:34:01.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T11:34:01.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T11:34:01.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T11:34:01.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T11:34:01.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T11:34:01.537 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T11:34:01.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T11:34:01.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T11:34:01.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T11:34:01.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T11:34:01.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9073d5d5-3cf8-4b85-9582-07c9e6950048 2024-04-25T11:34:01.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714044841487691, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T11:34:01.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T11:34:01.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714044841488704, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100032, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97914, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14131, "raw_average_key_size": 49, "raw_value_size": 90258, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714044841, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9073d5d5-3cf8-4b85-9582-07c9e6950048", "db_session_id": "CDXZ0B4A7UUZM9JDSI25", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T11:34:01.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714044841488785, "job": 1, "event": "recovery_finished"} 2024-04-25T11:34:01.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T11:34:01.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T11:34:01.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5600fb480000 2024-04-25T11:34:01.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: DB pointer 0x5600fb46a000 2024-04-25T11:34:01.538 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T11:34:01.539 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T11:34:01.539 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** DB Stats ** 2024-04-25T11:34:01.539 INFO:journalctl@ceph.mon.a.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T11:34:01.539 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T11:34:01.539 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T11:34:01.539 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T11:34:01.539 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T11:34:01.539 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T11:34:01.539 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T11:34:01.539 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-25T11:34:01.539 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** Compaction Stats [default] ** 2024-04-25T11:34:01.539 INFO:journalctl@ceph.mon.a.smithi083.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T11:34:01.539 INFO:journalctl@ceph.mon.a.smithi083.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T11:34:01.540 INFO:journalctl@ceph.mon.a.smithi083.stdout: L0 2/0 99.55 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 149.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T11:34:01.540 INFO:journalctl@ceph.mon.a.smithi083.stdout: Sum 2/0 99.55 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 149.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T11:34:01.540 INFO:journalctl@ceph.mon.a.smithi083.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 149.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T11:34:01.540 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-25T11:34:01.540 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** Compaction Stats [default] ** 2024-04-25T11:34:01.540 INFO:journalctl@ceph.mon.a.smithi083.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T11:34:01.540 INFO:journalctl@ceph.mon.a.smithi083.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T11:34:01.540 INFO:journalctl@ceph.mon.a.smithi083.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 149.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T11:34:01.540 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-25T11:34:01.540 INFO:journalctl@ceph.mon.a.smithi083.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T11:34:01.540 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-25T11:34:01.540 INFO:journalctl@ceph.mon.a.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T11:34:01.540 INFO:journalctl@ceph.mon.a.smithi083.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T11:34:01.540 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T11:34:01.541 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T11:34:01.541 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T11:34:01.541 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T11:34:01.541 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative compaction: 0.00 GB write, 31.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T11:34:01.541 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval compaction: 0.00 GB write, 31.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T11:34:01.541 INFO:journalctl@ceph.mon.a.smithi083.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T11:34:01.541 INFO:journalctl@ceph.mon.a.smithi083.stdout: Block cache BinnedLRUCache@0x5600fb36b1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-25T11:34:01.541 INFO:journalctl@ceph.mon.a.smithi083.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T11:34:01.541 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-25T11:34:01.541 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T11:34:01.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: starting mon.a rank 0 at public addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] at bind addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:34:01.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: mon.a@-1(???) e1 preinit fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:34:01.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: mon.a@-1(???).mds e1 new map 2024-04-25T11:34:01.541 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: mon.a@-1(???).mds e1 print_map 2024-04-25T11:34:01.542 INFO:journalctl@ceph.mon.a.smithi083.stdout: e1 2024-04-25T11:34:01.542 INFO:journalctl@ceph.mon.a.smithi083.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T11:34:01.542 INFO:journalctl@ceph.mon.a.smithi083.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T11:34:01.542 INFO:journalctl@ceph.mon.a.smithi083.stdout: legacy client fscid: -1 2024-04-25T11:34:01.542 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-25T11:34:01.542 INFO:journalctl@ceph.mon.a.smithi083.stdout: No filesystems configured 2024-04-25T11:34:01.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T11:34:01.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T11:34:01.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T11:34:01.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T11:34:01.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T11:34:01.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 bash[29311]: b35e79aacc4caa453c37b1077e06f233a5a06ac3bdda6cd43828836179f78708 2024-04-25T11:34:01.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 systemd[1]: Started Ceph mon.a for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:34:01.543 INFO:teuthology.orchestra.run.smithi083.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T11:34:01.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T11:34:01.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: monmap epoch 1 2024-04-25T11:34:01.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:34:01.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: last_changed 2024-04-25T11:33:42.458778+0000 2024-04-25T11:34:01.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: created 2024-04-25T11:33:42.458778+0000 2024-04-25T11:34:01.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: min_mon_release 19 (squid) 2024-04-25T11:34:01.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: election_strategy: 1 2024-04-25T11:34:01.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-25T11:34:01.774 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: fsmap 2024-04-25T11:34:01.775 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T11:34:01.775 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:01 smithi083 ceph-mon[29330]: mgrmap e1: no daemons active 2024-04-25T11:34:02.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.968+0000 7ff7399ba700 1 Processor -- start 2024-04-25T11:34:02.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.968+0000 7ff7399ba700 1 -- start start 2024-04-25T11:34:02.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.968+0000 7ff7399ba700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff734106ed0 0x7ff7341072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:02.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.968+0000 7ff7399ba700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff734063cc0 con 0x7ff734106ed0 2024-04-25T11:34:02.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.969+0000 7ff732ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff734106ed0 0x7ff7341072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:02.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.969+0000 7ff732ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff734106ed0 0x7ff7341072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56670/0 (socket says 172.21.15.83:56670) 2024-04-25T11:34:02.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.969+0000 7ff732ffd700 1 -- 172.21.15.83:0/3152524827 learned_addr learned my addr 172.21.15.83:0/3152524827 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:02.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.969+0000 7ff732ffd700 1 -- 172.21.15.83:0/3152524827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff734107830 con 0x7ff734106ed0 2024-04-25T11:34:02.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.969+0000 7ff732ffd700 1 --2- 172.21.15.83:0/3152524827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff734106ed0 0x7ff7341072f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff7240098a0 tx=0x7ff724009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401e71849ce4e62b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:02.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.969+0000 7ff7327fc700 1 -- 172.21.15.83:0/3152524827 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff724004160 con 0x7ff734106ed0 2024-04-25T11:34:02.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.969+0000 7ff7327fc700 1 -- 172.21.15.83:0/3152524827 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff724004300 con 0x7ff734106ed0 2024-04-25T11:34:02.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.969+0000 7ff7327fc700 1 -- 172.21.15.83:0/3152524827 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff724011af0 con 0x7ff734106ed0 2024-04-25T11:34:02.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.970+0000 7ff7399ba700 1 -- 172.21.15.83:0/3152524827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff734106ed0 msgr2=0x7ff7341072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.970+0000 7ff7399ba700 1 --2- 172.21.15.83:0/3152524827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff734106ed0 0x7ff7341072f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff7240098a0 tx=0x7ff724009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.970+0000 7ff7399ba700 1 -- 172.21.15.83:0/3152524827 shutdown_connections 2024-04-25T11:34:02.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.970+0000 7ff7399ba700 1 --2- 172.21.15.83:0/3152524827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff734106ed0 0x7ff7341072f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.970+0000 7ff7399ba700 1 -- 172.21.15.83:0/3152524827 >> 172.21.15.83:0/3152524827 conn(0x7ff734102ab0 msgr2=0x7ff734104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:02.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.970+0000 7ff7399ba700 1 -- 172.21.15.83:0/3152524827 shutdown_connections 2024-04-25T11:34:02.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.970+0000 7ff7399ba700 1 -- 172.21.15.83:0/3152524827 wait complete. 2024-04-25T11:34:02.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.970+0000 7ff7399ba700 1 Processor -- start 2024-04-25T11:34:02.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.970+0000 7ff7399ba700 1 -- start start 2024-04-25T11:34:02.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.971+0000 7ff7399ba700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff734106ed0 0x7ff73419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:02.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.971+0000 7ff7399ba700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7341087f0 con 0x7ff734106ed0 2024-04-25T11:34:02.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.971+0000 7ff732ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff734106ed0 0x7ff73419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:02.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.971+0000 7ff732ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff734106ed0 0x7ff73419e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56676/0 (socket says 172.21.15.83:56676) 2024-04-25T11:34:02.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.971+0000 7ff732ffd700 1 -- 172.21.15.83:0/261919443 learned_addr learned my addr 172.21.15.83:0/261919443 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:02.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.971+0000 7ff732ffd700 1 -- 172.21.15.83:0/261919443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73419eba0 con 0x7ff734106ed0 2024-04-25T11:34:02.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.971+0000 7ff732ffd700 1 --2- 172.21.15.83:0/261919443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff734106ed0 0x7ff73419e660 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff724011ac0 tx=0x7ff724011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:02.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.971+0000 7ff7389b8700 1 -- 172.21.15.83:0/261919443 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff724003b20 con 0x7ff734106ed0 2024-04-25T11:34:02.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.971+0000 7ff7389b8700 1 -- 172.21.15.83:0/261919443 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff72401a8c0 con 0x7ff734106ed0 2024-04-25T11:34:02.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.971+0000 7ff7399ba700 1 -- 172.21.15.83:0/261919443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73419ee30 con 0x7ff734106ed0 2024-04-25T11:34:02.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.972+0000 7ff7389b8700 1 -- 172.21.15.83:0/261919443 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7240248d0 con 0x7ff734106ed0 2024-04-25T11:34:02.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.972+0000 7ff7399ba700 1 -- 172.21.15.83:0/261919443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7341a1b20 con 0x7ff734106ed0 2024-04-25T11:34:02.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.972+0000 7ff7389b8700 1 -- 172.21.15.83:0/261919443 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff724022020 con 0x7ff734106ed0 2024-04-25T11:34:02.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.972+0000 7ff7389b8700 1 -- 172.21.15.83:0/261919443 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff7240193c0 con 0x7ff734106ed0 2024-04-25T11:34:02.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.972+0000 7ff7399ba700 1 -- 172.21.15.83:0/261919443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7340640a0 con 0x7ff734106ed0 2024-04-25T11:34:02.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:01.975+0000 7ff7389b8700 1 -- 172.21.15.83:0/261919443 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff72401aa70 con 0x7ff734106ed0 2024-04-25T11:34:02.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:02.043+0000 7ff7399ba700 1 -- 172.21.15.83:0/261919443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7ff73402e100 con 0x7ff734106ed0 2024-04-25T11:34:02.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:02.044+0000 7ff7389b8700 1 -- 172.21.15.83:0/261919443 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7ff72401a430 con 0x7ff734106ed0 2024-04-25T11:34:02.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:02.045+0000 7ff7399ba700 1 -- 172.21.15.83:0/261919443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff734106ed0 msgr2=0x7ff73419e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:02.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:02.045+0000 7ff7399ba700 1 --2- 172.21.15.83:0/261919443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff734106ed0 0x7ff73419e660 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff724011ac0 tx=0x7ff724011ba0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:02.045+0000 7ff7399ba700 1 -- 172.21.15.83:0/261919443 shutdown_connections 2024-04-25T11:34:02.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:02.045+0000 7ff7399ba700 1 --2- 172.21.15.83:0/261919443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff734106ed0 0x7ff73419e660 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:02.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:02.045+0000 7ff7399ba700 1 -- 172.21.15.83:0/261919443 >> 172.21.15.83:0/261919443 conn(0x7ff734102ab0 msgr2=0x7ff7341952f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:02.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:02.045+0000 7ff7399ba700 1 -- 172.21.15.83:0/261919443 shutdown_connections 2024-04-25T11:34:02.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:02.045+0000 7ff7399ba700 1 -- 172.21.15.83:0/261919443 wait complete. 2024-04-25T11:34:02.535 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T11:34:02.558 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:34:02.558 INFO:teuthology.orchestra.run.smithi083.stdout:Creating mgr... 2024-04-25T11:34:02.558 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T11:34:02.559 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T11:34:02.799 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.x 2024-04-25T11:34:02.799 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T11:34:02.948 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968.target.wants/ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@.service. 2024-04-25T11:34:03.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:03 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/261919443' entity='client.admin' 2024-04-25T11:34:03.579 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T11:34:03.579 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-04-25T11:34:03.587 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T11:34:03.587 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-25T11:34:03.587 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-04-25T11:34:03.587 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T11:34:03.595 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T11:34:03.595 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-04-25T11:34:03.602 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T11:34:03.602 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-25T11:34:03.602 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-04-25T11:34:03.602 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T11:34:03.602 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr to start... 2024-04-25T11:34:03.603 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr... 2024-04-25T11:34:03.706 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:03 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:03.594+0000 7f4e36f97200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:34:03.958 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:03 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:03.784+0000 7f4e36f97200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:34:03.958 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:03 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:03.888+0000 7f4e36f97200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:34:04.283 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:04 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/42365963' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T11:34:04.283 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:04 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:04.210+0000 7f4e36f97200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:34:04.283 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:04 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:04.294+0000 7f4e36f97200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:34:04.681 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-25T11:34:04.684 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:34:04.684 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "7fcf54c6-02f7-11ef-bc93-c7b262605968", 2024-04-25T11:34:04.684 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T11:34:04.684 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T11:34:04.684 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T11:34:04.684 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T11:34:04.684 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:04.684 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T11:34:04.685 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T11:34:04.685 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-04-25T11:34:04.685 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:34:04.685 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T11:34:04.685 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T11:34:04.685 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:34:04.685 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T11:34:04.685 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T11:34:04.685 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:04.685 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T11:34:04.685 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T11:34:04.685 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:04.686 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T11:34:04.686 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:04.694 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T11:34:04.694 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T11:34:04.694 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T11:34:04.694 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T11:34:04.694 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T11:34:04.694 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T11:34:04.694 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:04.694 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T11:34:04.695 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T11:34:04.695 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T11:34:04.695 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T11:34:04.695 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T11:34:04.695 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T11:34:04.695 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T11:34:04.695 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T11:34:04.695 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T11:34:04.695 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:04.695 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T11:34:04.695 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:04.695 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T11:34:04.695 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T11:34:04.695 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:04.696 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T11:34:04.696 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T11:34:04.696 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T11:34:04.696 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T11:34:04.696 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T11:34:04.696 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T11:34:04.696 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T11:34:04.696 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:34:04.696 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:34:04.696 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:04.696 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T11:34:04.696 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:04.696 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T11:33:56.406765+0000", 2024-04-25T11:34:04.696 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:34:04.697 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:04.697 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T11:34:04.697 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:34:04.697 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.003+0000 7f19dec64700 1 Processor -- start 2024-04-25T11:34:04.697 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.003+0000 7f19dec64700 1 -- start start 2024-04-25T11:34:04.697 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.003+0000 7f19dec64700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d81085c0 0x7f19d81089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:04.697 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.003+0000 7f19dec64700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19d8108fb0 con 0x7f19d81085c0 2024-04-25T11:34:04.697 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.004+0000 7f19dca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d81085c0 0x7f19d81089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:04.697 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.004+0000 7f19dca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d81085c0 0x7f19d81089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48028/0 (socket says 172.21.15.83:48028) 2024-04-25T11:34:04.697 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.004+0000 7f19dca00700 1 -- 172.21.15.83:0/1836412194 learned_addr learned my addr 172.21.15.83:0/1836412194 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:04.697 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.004+0000 7f19dca00700 1 -- 172.21.15.83:0/1836412194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d8109770 con 0x7f19d81085c0 2024-04-25T11:34:04.697 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.004+0000 7f19dca00700 1 --2- 172.21.15.83:0/1836412194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d81085c0 0x7f19d81089e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0098a0 tx=0x7f19cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37de655a74275844 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:04.698 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.005+0000 7f19d77fe700 1 -- 172.21.15.83:0/1836412194 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19cc004160 con 0x7f19d81085c0 2024-04-25T11:34:04.698 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.005+0000 7f19d77fe700 1 -- 172.21.15.83:0/1836412194 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f19cc004300 con 0x7f19d81085c0 2024-04-25T11:34:04.698 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.005+0000 7f19d77fe700 1 -- 172.21.15.83:0/1836412194 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19cc011af0 con 0x7f19d81085c0 2024-04-25T11:34:04.698 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.005+0000 7f19dec64700 1 -- 172.21.15.83:0/1836412194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d81085c0 msgr2=0x7f19d81089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:04.698 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.005+0000 7f19dec64700 1 --2- 172.21.15.83:0/1836412194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d81085c0 0x7f19d81089e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0098a0 tx=0x7f19cc009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:04.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.005+0000 7f19dec64700 1 -- 172.21.15.83:0/1836412194 shutdown_connections 2024-04-25T11:34:04.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.005+0000 7f19dec64700 1 --2- 172.21.15.83:0/1836412194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d81085c0 0x7f19d81089e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:04.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.005+0000 7f19dec64700 1 -- 172.21.15.83:0/1836412194 >> 172.21.15.83:0/1836412194 conn(0x7f19d8103ba0 msgr2=0x7f19d8105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:04.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.005+0000 7f19dec64700 1 -- 172.21.15.83:0/1836412194 shutdown_connections 2024-04-25T11:34:04.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.005+0000 7f19dec64700 1 -- 172.21.15.83:0/1836412194 wait complete. 2024-04-25T11:34:04.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.006+0000 7f19dec64700 1 Processor -- start 2024-04-25T11:34:04.699 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.006+0000 7f19dec64700 1 -- start start 2024-04-25T11:34:04.700 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.006+0000 7f19dec64700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d81085c0 0x7f19d819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:04.700 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.006+0000 7f19dec64700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19d810c9a0 con 0x7f19d81085c0 2024-04-25T11:34:04.700 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.006+0000 7f19dca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d81085c0 0x7f19d819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:04.700 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.006+0000 7f19dca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d81085c0 0x7f19d819c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48038/0 (socket says 172.21.15.83:48038) 2024-04-25T11:34:04.700 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.006+0000 7f19dca00700 1 -- 172.21.15.83:0/42365963 learned_addr learned my addr 172.21.15.83:0/42365963 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:04.700 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.007+0000 7f19dca00700 1 -- 172.21.15.83:0/42365963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d819ca10 con 0x7f19d81085c0 2024-04-25T11:34:04.700 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.007+0000 7f19dca00700 1 --2- 172.21.15.83:0/42365963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d81085c0 0x7f19d819c4d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f19cc003f40 tx=0x7f19cc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:04.700 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.007+0000 7f19d5ffb700 1 -- 172.21.15.83:0/42365963 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19cc0038b0 con 0x7f19d81085c0 2024-04-25T11:34:04.700 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.007+0000 7f19d5ffb700 1 -- 172.21.15.83:0/42365963 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f19cc003a50 con 0x7f19d81085c0 2024-04-25T11:34:04.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.007+0000 7f19dec64700 1 -- 172.21.15.83:0/42365963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d819cca0 con 0x7f19d81085c0 2024-04-25T11:34:04.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.007+0000 7f19d5ffb700 1 -- 172.21.15.83:0/42365963 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19cc01a5c0 con 0x7f19d81085c0 2024-04-25T11:34:04.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.007+0000 7f19dec64700 1 -- 172.21.15.83:0/42365963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d819f990 con 0x7f19d81085c0 2024-04-25T11:34:04.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.007+0000 7f19d5ffb700 1 -- 172.21.15.83:0/42365963 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f19cc01ac50 con 0x7f19d81085c0 2024-04-25T11:34:04.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.007+0000 7f19d5ffb700 1 -- 172.21.15.83:0/42365963 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f19cc019700 con 0x7f19d81085c0 2024-04-25T11:34:04.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.008+0000 7f19dec64700 1 -- 172.21.15.83:0/42365963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d810cb20 con 0x7f19d81085c0 2024-04-25T11:34:04.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.010+0000 7f19d5ffb700 1 -- 172.21.15.83:0/42365963 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f19cc0199a0 con 0x7f19d81085c0 2024-04-25T11:34:04.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.081+0000 7f19dec64700 1 -- 172.21.15.83:0/42365963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f19d80640a0 con 0x7f19d81085c0 2024-04-25T11:34:04.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.082+0000 7f19d5ffb700 1 -- 172.21.15.83:0/42365963 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f19d80640a0 con 0x7f19d81085c0 2024-04-25T11:34:04.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.083+0000 7f19dec64700 1 -- 172.21.15.83:0/42365963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d81085c0 msgr2=0x7f19d819c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:04.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.083+0000 7f19dec64700 1 --2- 172.21.15.83:0/42365963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d81085c0 0x7f19d819c4d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f19cc003f40 tx=0x7f19cc011890 comp rx=0 tx=0).stop 2024-04-25T11:34:04.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.083+0000 7f19dec64700 1 -- 172.21.15.83:0/42365963 shutdown_connections 2024-04-25T11:34:04.701 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.083+0000 7f19dec64700 1 --2- 172.21.15.83:0/42365963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19d81085c0 0x7f19d819c4d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:04.702 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.083+0000 7f19dec64700 1 -- 172.21.15.83:0/42365963 >> 172.21.15.83:0/42365963 conn(0x7f19d8103ba0 msgr2=0x7f19d8104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:04.702 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.083+0000 7f19dec64700 1 -- 172.21.15.83:0/42365963 shutdown_connections 2024-04-25T11:34:04.702 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:04.083+0000 7f19dec64700 1 -- 172.21.15.83:0/42365963 wait complete. 2024-04-25T11:34:04.702 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (1/15)... 2024-04-25T11:34:05.791 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:05 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:05.470+0000 7f4e36f97200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:34:05.791 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:05 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:05.637+0000 7f4e36f97200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:34:05.791 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:05 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:05.802+0000 7f4e36f97200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:34:06.363 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:06 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:06.085+0000 7f4e36f97200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:34:06.363 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:06 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:06.176+0000 7f4e36f97200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:34:06.363 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:06 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:06.374+0000 7f4e36f97200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:34:06.657 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:06 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:06.565+0000 7f4e36f97200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:34:07.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:07 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1312605190' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T11:34:07.407 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:07 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:07.184+0000 7f4e36f97200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:34:07.907 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:07 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:07.640+0000 7f4e36f97200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:34:07.907 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:07 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:07.736+0000 7f4e36f97200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:34:07.948 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-25T11:34:07.948 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:34:07.948 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "7fcf54c6-02f7-11ef-bc93-c7b262605968", 2024-04-25T11:34:07.948 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T11:34:07.948 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T11:34:07.948 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T11:34:07.948 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T11:34:07.948 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:07.948 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T11:34:07.948 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T11:34:07.948 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-04-25T11:34:07.948 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:34:07.949 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T11:34:07.949 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T11:34:07.949 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:34:07.949 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-25T11:34:07.949 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T11:34:07.949 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:07.949 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T11:34:07.949 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T11:34:07.949 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:07.949 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T11:34:07.949 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:07.949 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T11:34:07.949 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T11:34:07.949 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T11:34:07.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T11:34:07.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T11:34:07.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T11:34:07.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:07.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T11:34:07.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T11:34:07.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T11:34:07.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T11:34:07.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T11:34:07.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T11:34:07.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T11:34:07.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T11:34:07.950 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T11:34:07.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:07.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T11:34:07.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:07.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T11:34:07.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T11:34:07.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:07.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T11:34:07.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T11:34:07.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T11:34:07.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T11:34:07.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T11:34:07.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T11:34:07.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T11:34:07.951 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:34:07.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:34:07.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:07.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T11:34:07.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:07.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T11:33:56.406765+0000", 2024-04-25T11:34:07.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:34:07.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:07.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T11:34:07.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:34:07.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.160+0000 7f4bdeb3d700 1 Processor -- start 2024-04-25T11:34:07.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.160+0000 7f4bdeb3d700 1 -- start start 2024-04-25T11:34:07.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.161+0000 7f4bdeb3d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd807de70 0x7f4bd807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.161+0000 7f4bdeb3d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd807c8f0 con 0x7f4bd807de70 2024-04-25T11:34:07.952 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.161+0000 7f4bdc8d9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd807de70 0x7f4bd807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.161+0000 7f4bdc8d9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd807de70 0x7f4bd807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48042/0 (socket says 172.21.15.83:48042) 2024-04-25T11:34:07.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.161+0000 7f4bdc8d9700 1 -- 172.21.15.83:0/1656034622 learned_addr learned my addr 172.21.15.83:0/1656034622 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:07.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.161+0000 7f4bdc8d9700 1 -- 172.21.15.83:0/1656034622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd807ca70 con 0x7f4bd807de70 2024-04-25T11:34:07.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.161+0000 7f4bdc8d9700 1 --2- 172.21.15.83:0/1656034622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd807de70 0x7f4bd807c320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0098a0 tx=0x7f4bcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ab1da97da7abc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:07.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.161+0000 7f4bd77fe700 1 -- 172.21.15.83:0/1656034622 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bcc004160 con 0x7f4bd807de70 2024-04-25T11:34:07.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.161+0000 7f4bd77fe700 1 -- 172.21.15.83:0/1656034622 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4bcc004300 con 0x7f4bd807de70 2024-04-25T11:34:07.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.161+0000 7f4bd77fe700 1 -- 172.21.15.83:0/1656034622 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bcc011af0 con 0x7f4bd807de70 2024-04-25T11:34:07.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.162+0000 7f4bdeb3d700 1 -- 172.21.15.83:0/1656034622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd807de70 msgr2=0x7f4bd807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:07.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.162+0000 7f4bdeb3d700 1 --2- 172.21.15.83:0/1656034622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd807de70 0x7f4bd807c320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0098a0 tx=0x7f4bcc009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.162+0000 7f4bdeb3d700 1 -- 172.21.15.83:0/1656034622 shutdown_connections 2024-04-25T11:34:07.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.162+0000 7f4bdeb3d700 1 --2- 172.21.15.83:0/1656034622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd807de70 0x7f4bd807c320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.162+0000 7f4bdeb3d700 1 -- 172.21.15.83:0/1656034622 >> 172.21.15.83:0/1656034622 conn(0x7f4bd8103bd0 msgr2=0x7f4bd8105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:07.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.162+0000 7f4bdeb3d700 1 -- 172.21.15.83:0/1656034622 shutdown_connections 2024-04-25T11:34:07.953 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.162+0000 7f4bdeb3d700 1 -- 172.21.15.83:0/1656034622 wait complete. 2024-04-25T11:34:07.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.163+0000 7f4bdeb3d700 1 Processor -- start 2024-04-25T11:34:07.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.163+0000 7f4bdeb3d700 1 -- start start 2024-04-25T11:34:07.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.163+0000 7f4bdeb3d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd807de70 0x7f4bd819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:07.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.163+0000 7f4bdeb3d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd810eae0 con 0x7f4bd807de70 2024-04-25T11:34:07.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.163+0000 7f4bdc8d9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd807de70 0x7f4bd819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:07.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.163+0000 7f4bdc8d9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd807de70 0x7f4bd819e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48056/0 (socket says 172.21.15.83:48056) 2024-04-25T11:34:07.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.163+0000 7f4bdc8d9700 1 -- 172.21.15.83:0/1312605190 learned_addr learned my addr 172.21.15.83:0/1312605190 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:07.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.163+0000 7f4bdc8d9700 1 -- 172.21.15.83:0/1312605190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd819eb90 con 0x7f4bd807de70 2024-04-25T11:34:07.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.164+0000 7f4bdc8d9700 1 --2- 172.21.15.83:0/1312605190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd807de70 0x7f4bd819e650 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc011ac0 tx=0x7f4bcc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:07.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.164+0000 7f4bd5ffb700 1 -- 172.21.15.83:0/1312605190 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bcc003b20 con 0x7f4bd807de70 2024-04-25T11:34:07.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.164+0000 7f4bdeb3d700 1 -- 172.21.15.83:0/1312605190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd819ee20 con 0x7f4bd807de70 2024-04-25T11:34:07.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.164+0000 7f4bd5ffb700 1 -- 172.21.15.83:0/1312605190 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4bcc01a8c0 con 0x7f4bd807de70 2024-04-25T11:34:07.954 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.164+0000 7f4bd5ffb700 1 -- 172.21.15.83:0/1312605190 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bcc0248d0 con 0x7f4bd807de70 2024-04-25T11:34:07.955 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.164+0000 7f4bdeb3d700 1 -- 172.21.15.83:0/1312605190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd81a1b10 con 0x7f4bd807de70 2024-04-25T11:34:07.955 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.164+0000 7f4bd5ffb700 1 -- 172.21.15.83:0/1312605190 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4bcc022020 con 0x7f4bd807de70 2024-04-25T11:34:07.955 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.164+0000 7f4bd5ffb700 1 -- 172.21.15.83:0/1312605190 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4bcc00b6f0 con 0x7f4bd807de70 2024-04-25T11:34:07.955 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.165+0000 7f4bdeb3d700 1 -- 172.21.15.83:0/1312605190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd80640a0 con 0x7f4bd807de70 2024-04-25T11:34:07.955 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.167+0000 7f4bd5ffb700 1 -- 172.21.15.83:0/1312605190 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4bcc024e00 con 0x7f4bd807de70 2024-04-25T11:34:07.955 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.286+0000 7f4bdeb3d700 1 -- 172.21.15.83:0/1312605190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4bd81a2000 con 0x7f4bd807de70 2024-04-25T11:34:07.955 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.286+0000 7f4bd5ffb700 1 -- 172.21.15.83:0/1312605190 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4bcc01aa70 con 0x7f4bd807de70 2024-04-25T11:34:07.955 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.287+0000 7f4bdeb3d700 1 -- 172.21.15.83:0/1312605190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd807de70 msgr2=0x7f4bd819e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:07.955 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.287+0000 7f4bdeb3d700 1 --2- 172.21.15.83:0/1312605190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd807de70 0x7f4bd819e650 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc011ac0 tx=0x7f4bcc011ba0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.955 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.287+0000 7f4bdeb3d700 1 -- 172.21.15.83:0/1312605190 shutdown_connections 2024-04-25T11:34:07.955 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.287+0000 7f4bdeb3d700 1 --2- 172.21.15.83:0/1312605190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd807de70 0x7f4bd819e650 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:07.955 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.287+0000 7f4bdeb3d700 1 -- 172.21.15.83:0/1312605190 >> 172.21.15.83:0/1312605190 conn(0x7f4bd8103bd0 msgr2=0x7f4bd8104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:07.955 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.287+0000 7f4bdeb3d700 1 -- 172.21.15.83:0/1312605190 shutdown_connections 2024-04-25T11:34:07.955 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:07.287+0000 7f4bdeb3d700 1 -- 172.21.15.83:0/1312605190 wait complete. 2024-04-25T11:34:07.956 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (2/15)... 2024-04-25T11:34:08.407 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:07 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:07.939+0000 7f4e36f97200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:34:08.789 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:08 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:08.468+0000 7f4e36f97200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:34:08.789 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:08 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:08.559+0000 7f4e36f97200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:34:09.132 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:08 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:08.800+0000 7f4e36f97200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:34:09.132 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:08 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:08.932+0000 7f4e36f97200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:34:09.132 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:09 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:09.028+0000 7f4e36f97200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:34:09.132 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:09 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:09.143+0000 7f4e36f97200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:34:09.907 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:09 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:09.445+0000 7f4e36f97200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:34:10.307 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:10 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:10.045+0000 7f4e36f97200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:34:10.307 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:10 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:10.266+0000 7f4e36f97200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:34:10.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:10 smithi083 ceph-mon[29330]: Activating manager daemon x 2024-04-25T11:34:10.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:10 smithi083 ceph-mon[29330]: mgrmap e2: x(active, starting, since 0.00317204s) 2024-04-25T11:34:10.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:10 smithi083 ceph-mon[29330]: from='mgr.14100 172.21.15.83:0/4182013114' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T11:34:10.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:10 smithi083 ceph-mon[29330]: from='mgr.14100 172.21.15.83:0/4182013114' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T11:34:10.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:10 smithi083 ceph-mon[29330]: from='mgr.14100 172.21.15.83:0/4182013114' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T11:34:10.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:10 smithi083 ceph-mon[29330]: from='mgr.14100 172.21.15.83:0/4182013114' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:34:10.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:10 smithi083 ceph-mon[29330]: from='mgr.14100 172.21.15.83:0/4182013114' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T11:34:10.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:10 smithi083 ceph-mon[29330]: Manager daemon x is now available 2024-04-25T11:34:10.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:10 smithi083 ceph-mon[29330]: from='mgr.14100 172.21.15.83:0/4182013114' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T11:34:10.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:10 smithi083 ceph-mon[29330]: from='mgr.14100 172.21.15.83:0/4182013114' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T11:34:10.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:10 smithi083 ceph-mon[29330]: from='mgr.14100 172.21.15.83:0/4182013114' entity='mgr.x' 2024-04-25T11:34:10.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:10 smithi083 ceph-mon[29330]: from='mgr.14100 172.21.15.83:0/4182013114' entity='mgr.x' 2024-04-25T11:34:10.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:10 smithi083 ceph-mon[29330]: from='mgr.14100 172.21.15.83:0/4182013114' entity='mgr.x' 2024-04-25T11:34:11.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-25T11:34:11.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:34:11.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "7fcf54c6-02f7-11ef-bc93-c7b262605968", 2024-04-25T11:34:11.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T11:34:11.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T11:34:11.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T11:34:11.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T11:34:11.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:11.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T11:34:11.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T11:34:11.055 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-04-25T11:34:11.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:34:11.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T11:34:11.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T11:34:11.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:34:11.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-25T11:34:11.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T11:34:11.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:11.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T11:34:11.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T11:34:11.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:11.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T11:34:11.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:11.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T11:34:11.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T11:34:11.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T11:34:11.064 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T11:34:11.064 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T11:34:11.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T11:34:11.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:11.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T11:34:11.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T11:34:11.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T11:34:11.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T11:34:11.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T11:34:11.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T11:34:11.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T11:34:11.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T11:34:11.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T11:34:11.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:11.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T11:34:11.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:11.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T11:34:11.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T11:34:11.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:11.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T11:34:11.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T11:34:11.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T11:34:11.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T11:34:11.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T11:34:11.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T11:34:11.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T11:34:11.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:34:11.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:34:11.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T11:34:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T11:33:56.406765+0000", 2024-04-25T11:34:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:34:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T11:34:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:34:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.388+0000 7fc6c8e3f700 1 Processor -- start 2024-04-25T11:34:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.388+0000 7fc6c8e3f700 1 -- start start 2024-04-25T11:34:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.388+0000 7fc6c8e3f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c4106c60 0x7fc6c4107080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.388+0000 7fc6c8e3f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c4107650 con 0x7fc6c4106c60 2024-04-25T11:34:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.388+0000 7fc6c259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c4106c60 0x7fc6c4107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.388+0000 7fc6c259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c4106c60 0x7fc6c4107080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48124/0 (socket says 172.21.15.83:48124) 2024-04-25T11:34:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.388+0000 7fc6c259c700 1 -- 172.21.15.83:0/186004489 learned_addr learned my addr 172.21.15.83:0/186004489 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.388+0000 7fc6c259c700 1 -- 172.21.15.83:0/186004489 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c4107e60 con 0x7fc6c4106c60 2024-04-25T11:34:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.389+0000 7fc6c259c700 1 --2- 172.21.15.83:0/186004489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c4106c60 0x7fc6c4107080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40098a0 tx=0x7fc6b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fb503e0cb8b645 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.389+0000 7fc6c159a700 1 -- 172.21.15.83:0/186004489 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6b4004160 con 0x7fc6c4106c60 2024-04-25T11:34:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.389+0000 7fc6c159a700 1 -- 172.21.15.83:0/186004489 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc6b4004300 con 0x7fc6c4106c60 2024-04-25T11:34:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.389+0000 7fc6c159a700 1 -- 172.21.15.83:0/186004489 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6b4011af0 con 0x7fc6c4106c60 2024-04-25T11:34:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.389+0000 7fc6c8e3f700 1 -- 172.21.15.83:0/186004489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c4106c60 msgr2=0x7fc6c4107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.389+0000 7fc6c8e3f700 1 --2- 172.21.15.83:0/186004489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c4106c60 0x7fc6c4107080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40098a0 tx=0x7fc6b4009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.389+0000 7fc6c8e3f700 1 -- 172.21.15.83:0/186004489 shutdown_connections 2024-04-25T11:34:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.389+0000 7fc6c8e3f700 1 --2- 172.21.15.83:0/186004489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c4106c60 0x7fc6c4107080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.389+0000 7fc6c8e3f700 1 -- 172.21.15.83:0/186004489 >> 172.21.15.83:0/186004489 conn(0x7fc6c4102200 msgr2=0x7fc6c4104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.390+0000 7fc6c8e3f700 1 -- 172.21.15.83:0/186004489 shutdown_connections 2024-04-25T11:34:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.390+0000 7fc6c8e3f700 1 -- 172.21.15.83:0/186004489 wait complete. 2024-04-25T11:34:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.390+0000 7fc6c8e3f700 1 Processor -- start 2024-04-25T11:34:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.390+0000 7fc6c8e3f700 1 -- start start 2024-04-25T11:34:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.390+0000 7fc6c8e3f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c4106c60 0x7fc6c419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.390+0000 7fc6c8e3f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c410b090 con 0x7fc6c4106c60 2024-04-25T11:34:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.391+0000 7fc6c259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c4106c60 0x7fc6c419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.391+0000 7fc6c259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c4106c60 0x7fc6c419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48136/0 (socket says 172.21.15.83:48136) 2024-04-25T11:34:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.391+0000 7fc6c259c700 1 -- 172.21.15.83:0/2854772144 learned_addr learned my addr 172.21.15.83:0/2854772144 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.391+0000 7fc6c259c700 1 -- 172.21.15.83:0/2854772144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c419b980 con 0x7fc6c4106c60 2024-04-25T11:34:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.391+0000 7fc6c259c700 1 --2- 172.21.15.83:0/2854772144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c4106c60 0x7fc6c419b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4003f40 tx=0x7fc6b4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.391+0000 7fc6b37fe700 1 -- 172.21.15.83:0/2854772144 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6b40038b0 con 0x7fc6c4106c60 2024-04-25T11:34:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.391+0000 7fc6b37fe700 1 -- 172.21.15.83:0/2854772144 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc6b4003a50 con 0x7fc6c4106c60 2024-04-25T11:34:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.391+0000 7fc6c8e3f700 1 -- 172.21.15.83:0/2854772144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c419bc10 con 0x7fc6c4106c60 2024-04-25T11:34:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.391+0000 7fc6b37fe700 1 -- 172.21.15.83:0/2854772144 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6b401a5c0 con 0x7fc6c4106c60 2024-04-25T11:34:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.391+0000 7fc6c8e3f700 1 -- 172.21.15.83:0/2854772144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c419e8b0 con 0x7fc6c4106c60 2024-04-25T11:34:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.392+0000 7fc6b37fe700 1 -- 172.21.15.83:0/2854772144 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 2) ==== 46314+0+0 (secure 0 0 0) 0x7fc6b401a8c0 con 0x7fc6c4106c60 2024-04-25T11:34:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.392+0000 7fc6b37fe700 1 -- 172.21.15.83:0/2854772144 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc6b404d810 con 0x7fc6c4106c60 2024-04-25T11:34:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.392+0000 7fc6c8e3f700 1 -- 172.21.15.83:0/2854772144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c410b210 con 0x7fc6c4106c60 2024-04-25T11:34:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.394+0000 7fc6b37fe700 1 -- 172.21.15.83:0/2854772144 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc6b402d3f0 con 0x7fc6c4106c60 2024-04-25T11:34:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.464+0000 7fc6c8e3f700 1 -- 172.21.15.83:0/2854772144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc6c402e0d0 con 0x7fc6c4106c60 2024-04-25T11:34:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.464+0000 7fc6b37fe700 1 -- 172.21.15.83:0/2854772144 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc6b4031020 con 0x7fc6c4106c60 2024-04-25T11:34:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.465+0000 7fc6c8e3f700 1 -- 172.21.15.83:0/2854772144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c4106c60 msgr2=0x7fc6c419b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.465+0000 7fc6c8e3f700 1 --2- 172.21.15.83:0/2854772144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c4106c60 0x7fc6c419b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4003f40 tx=0x7fc6b4011890 comp rx=0 tx=0).stop 2024-04-25T11:34:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.466+0000 7fc6c8e3f700 1 -- 172.21.15.83:0/2854772144 shutdown_connections 2024-04-25T11:34:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.466+0000 7fc6c8e3f700 1 --2- 172.21.15.83:0/2854772144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc6c4106c60 0x7fc6c419b440 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.466+0000 7fc6c8e3f700 1 -- 172.21.15.83:0/2854772144 >> 172.21.15.83:0/2854772144 conn(0x7fc6c4102200 msgr2=0x7fc6c4104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.466+0000 7fc6c8e3f700 1 -- 172.21.15.83:0/2854772144 shutdown_connections 2024-04-25T11:34:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:10.466+0000 7fc6c8e3f700 1 -- 172.21.15.83:0/2854772144 wait complete. 2024-04-25T11:34:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (3/15)... 2024-04-25T11:34:11.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:11 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2854772144' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T11:34:11.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:11 smithi083 ceph-mon[29330]: mgrmap e3: x(active, since 1.00703s) 2024-04-25T11:34:13.517 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:13 smithi083 ceph-mon[29330]: mgrmap e4: x(active, since 2s) 2024-04-25T11:34:14.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-25T11:34:14.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:34:14.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "7fcf54c6-02f7-11ef-bc93-c7b262605968", 2024-04-25T11:34:14.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T11:34:14.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T11:34:14.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T11:34:14.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T11:34:14.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:14.360 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T11:34:14.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T11:34:14.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-04-25T11:34:14.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:34:14.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T11:34:14.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T11:34:14.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:34:14.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-25T11:34:14.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T11:34:14.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:14.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T11:34:14.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T11:34:14.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:14.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T11:34:14.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:14.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T11:34:14.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T11:34:14.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T11:34:14.370 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T11:34:14.370 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T11:34:14.370 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T11:34:14.370 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:14.370 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T11:34:14.370 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T11:34:14.370 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T11:34:14.370 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T11:34:14.371 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T11:34:14.371 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T11:34:14.371 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T11:34:14.371 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T11:34:14.371 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T11:34:14.371 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:14.371 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T11:34:14.371 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:14.371 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T11:34:14.371 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T11:34:14.371 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:14.371 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T11:34:14.371 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T11:34:14.371 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T11:34:14.371 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T11:34:14.372 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T11:34:14.372 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T11:34:14.372 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T11:34:14.372 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-25T11:34:14.372 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:34:14.372 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:14.372 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T11:34:14.372 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T11:34:14.372 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T11:33:56.406765+0000", 2024-04-25T11:34:14.372 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T11:34:14.372 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-25T11:34:14.372 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T11:34:14.372 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:34:14.372 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.528+0000 7fbeb6f39700 1 Processor -- start 2024-04-25T11:34:14.373 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.528+0000 7fbeb6f39700 1 -- start start 2024-04-25T11:34:14.373 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.528+0000 7fbeb6f39700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbeb0108600 0x7fbeb0108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:14.373 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.528+0000 7fbeb6f39700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0108ff0 con 0x7fbeb0108600 2024-04-25T11:34:14.373 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.528+0000 7fbeb4cd5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbeb0108600 0x7fbeb0108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:14.373 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.528+0000 7fbeb4cd5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbeb0108600 0x7fbeb0108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57532/0 (socket says 172.21.15.83:57532) 2024-04-25T11:34:14.373 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.528+0000 7fbeb4cd5700 1 -- 172.21.15.83:0/3351133059 learned_addr learned my addr 172.21.15.83:0/3351133059 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:14.373 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.529+0000 7fbeb4cd5700 1 -- 172.21.15.83:0/3351133059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb01097b0 con 0x7fbeb0108600 2024-04-25T11:34:14.373 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.529+0000 7fbeb4cd5700 1 --2- 172.21.15.83:0/3351133059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbeb0108600 0x7fbeb0108a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbea40098a0 tx=0x7fbea4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47143c67e6c3fd92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:14.373 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.529+0000 7fbeaf7fe700 1 -- 172.21.15.83:0/3351133059 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea4004160 con 0x7fbeb0108600 2024-04-25T11:34:14.373 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.529+0000 7fbeaf7fe700 1 -- 172.21.15.83:0/3351133059 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbea4004300 con 0x7fbeb0108600 2024-04-25T11:34:14.373 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.529+0000 7fbeaf7fe700 1 -- 172.21.15.83:0/3351133059 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea4011af0 con 0x7fbeb0108600 2024-04-25T11:34:14.373 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.529+0000 7fbeb6f39700 1 -- 172.21.15.83:0/3351133059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbeb0108600 msgr2=0x7fbeb0108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:14.373 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.529+0000 7fbeb6f39700 1 --2- 172.21.15.83:0/3351133059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbeb0108600 0x7fbeb0108a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbea40098a0 tx=0x7fbea4009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:14.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.529+0000 7fbeb6f39700 1 -- 172.21.15.83:0/3351133059 shutdown_connections 2024-04-25T11:34:14.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.529+0000 7fbeb6f39700 1 --2- 172.21.15.83:0/3351133059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbeb0108600 0x7fbeb0108a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:14.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.529+0000 7fbeb6f39700 1 -- 172.21.15.83:0/3351133059 >> 172.21.15.83:0/3351133059 conn(0x7fbeb0103bc0 msgr2=0x7fbeb0105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:14.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.529+0000 7fbeb6f39700 1 -- 172.21.15.83:0/3351133059 shutdown_connections 2024-04-25T11:34:14.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.529+0000 7fbeb6f39700 1 -- 172.21.15.83:0/3351133059 wait complete. 2024-04-25T11:34:14.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.530+0000 7fbeb6f39700 1 Processor -- start 2024-04-25T11:34:14.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.530+0000 7fbeb6f39700 1 -- start start 2024-04-25T11:34:14.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.530+0000 7fbeb6f39700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbeb0108600 0x7fbeb019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:14.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.530+0000 7fbeb6f39700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb010c9e0 con 0x7fbeb0108600 2024-04-25T11:34:14.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.530+0000 7fbeb4cd5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbeb0108600 0x7fbeb019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:14.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.530+0000 7fbeb4cd5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbeb0108600 0x7fbeb019c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57540/0 (socket says 172.21.15.83:57540) 2024-04-25T11:34:14.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.530+0000 7fbeb4cd5700 1 -- 172.21.15.83:0/543123631 learned_addr learned my addr 172.21.15.83:0/543123631 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:14.374 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.531+0000 7fbeb4cd5700 1 -- 172.21.15.83:0/543123631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb019c9d0 con 0x7fbeb0108600 2024-04-25T11:34:14.375 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.531+0000 7fbeb4cd5700 1 --2- 172.21.15.83:0/543123631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbeb0108600 0x7fbeb019c490 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbea4003f40 tx=0x7fbea4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:14.375 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.531+0000 7fbeadffb700 1 -- 172.21.15.83:0/543123631 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea40038b0 con 0x7fbeb0108600 2024-04-25T11:34:14.375 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.531+0000 7fbeadffb700 1 -- 172.21.15.83:0/543123631 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbea4003a50 con 0x7fbeb0108600 2024-04-25T11:34:14.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.531+0000 7fbeb6f39700 1 -- 172.21.15.83:0/543123631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb019cc60 con 0x7fbeb0108600 2024-04-25T11:34:14.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.531+0000 7fbeadffb700 1 -- 172.21.15.83:0/543123631 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbea401a5c0 con 0x7fbeb0108600 2024-04-25T11:34:14.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.531+0000 7fbeb6f39700 1 -- 172.21.15.83:0/543123631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb019f950 con 0x7fbeb0108600 2024-04-25T11:34:14.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.532+0000 7fbeadffb700 1 -- 172.21.15.83:0/543123631 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fbea401a8c0 con 0x7fbeb0108600 2024-04-25T11:34:14.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.532+0000 7fbeadffb700 1 --2- 172.21.15.83:0/543123631 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7fbea0039510 0x7fbea003b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:14.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.532+0000 7fbeb6f39700 1 -- 172.21.15.83:0/543123631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb010cb60 con 0x7fbeb0108600 2024-04-25T11:34:14.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.532+0000 7fbeadffb700 1 -- 172.21.15.83:0/543123631 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbea404dba0 con 0x7fbeb0108600 2024-04-25T11:34:14.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.532+0000 7fbeaffff700 1 --2- 172.21.15.83:0/543123631 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7fbea0039510 0x7fbea003b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:14.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.533+0000 7fbeaffff700 1 --2- 172.21.15.83:0/543123631 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7fbea0039510 0x7fbea003b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c006fd0 tx=0x7fbe9c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:14.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.538+0000 7fbeadffb700 1 -- 172.21.15.83:0/543123631 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbea4019c70 con 0x7fbeb0108600 2024-04-25T11:34:14.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.812+0000 7fbeb6f39700 1 -- 172.21.15.83:0/543123631 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbeb019fb20 con 0x7fbeb0108600 2024-04-25T11:34:14.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.812+0000 7fbeadffb700 1 -- 172.21.15.83:0/543123631 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fbea404d5c0 con 0x7fbeb0108600 2024-04-25T11:34:14.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.815+0000 7fbeb6f39700 1 -- 172.21.15.83:0/543123631 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7fbea0039510 msgr2=0x7fbea003b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:14.376 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.815+0000 7fbeb6f39700 1 --2- 172.21.15.83:0/543123631 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7fbea0039510 0x7fbea003b9d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c006fd0 tx=0x7fbe9c006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:14.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.815+0000 7fbeb6f39700 1 -- 172.21.15.83:0/543123631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbeb0108600 msgr2=0x7fbeb019c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:14.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.815+0000 7fbeb6f39700 1 --2- 172.21.15.83:0/543123631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbeb0108600 0x7fbeb019c490 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbea4003f40 tx=0x7fbea4011890 comp rx=0 tx=0).stop 2024-04-25T11:34:14.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.815+0000 7fbeb6f39700 1 -- 172.21.15.83:0/543123631 shutdown_connections 2024-04-25T11:34:14.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.815+0000 7fbeb6f39700 1 --2- 172.21.15.83:0/543123631 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7fbea0039510 0x7fbea003b9d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:14.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.815+0000 7fbeb6f39700 1 --2- 172.21.15.83:0/543123631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbeb0108600 0x7fbeb019c490 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:14.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.815+0000 7fbeb6f39700 1 -- 172.21.15.83:0/543123631 >> 172.21.15.83:0/543123631 conn(0x7fbeb0103bc0 msgr2=0x7fbeb01046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:14.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.815+0000 7fbeb6f39700 1 -- 172.21.15.83:0/543123631 shutdown_connections 2024-04-25T11:34:14.377 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:13.815+0000 7fbeb6f39700 1 -- 172.21.15.83:0/543123631 wait complete. 2024-04-25T11:34:14.377 INFO:teuthology.orchestra.run.smithi083.stdout:mgr is available 2024-04-25T11:34:14.484 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:14 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/543123631' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T11:34:15.539 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-25T11:34:15.539 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T11:34:15.539 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout fsid = 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:34:15.539 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T11:34:15.539 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T11:34:15.539 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T11:34:15.539 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T11:34:15.539 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T11:34:15.539 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-25T11:34:15.540 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T11:34:15.540 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T11:34:15.540 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-25T11:34:15.540 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T11:34:15.540 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T11:34:15.540 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T11:34:15.540 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T11:34:15.540 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T11:34:15.540 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T11:34:15.540 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T11:34:15.540 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.815+0000 7f49c7942700 1 Processor -- start 2024-04-25T11:34:15.540 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.815+0000 7f49c7942700 1 -- start start 2024-04-25T11:34:15.540 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.815+0000 7f49c7942700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c0109b90 0x7f49c0109fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:15.540 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.815+0000 7f49c7942700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49c010a580 con 0x7f49c0109b90 2024-04-25T11:34:15.541 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.816+0000 7f49c56de700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c0109b90 0x7f49c0109fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:15.541 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.816+0000 7f49c56de700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c0109b90 0x7f49c0109fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57542/0 (socket says 172.21.15.83:57542) 2024-04-25T11:34:15.541 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.816+0000 7f49c56de700 1 -- 172.21.15.83:0/2704254131 learned_addr learned my addr 172.21.15.83:0/2704254131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:15.541 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.816+0000 7f49c56de700 1 -- 172.21.15.83:0/2704254131 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c010a700 con 0x7f49c0109b90 2024-04-25T11:34:15.541 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.816+0000 7f49c56de700 1 --2- 172.21.15.83:0/2704254131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c0109b90 0x7f49c0109fb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0098a0 tx=0x7f49bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8451d3e21f163899 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:15.541 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.816+0000 7f49b7fff700 1 -- 172.21.15.83:0/2704254131 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49bc004160 con 0x7f49c0109b90 2024-04-25T11:34:15.541 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.816+0000 7f49b7fff700 1 -- 172.21.15.83:0/2704254131 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f49bc004300 con 0x7f49c0109b90 2024-04-25T11:34:15.541 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.816+0000 7f49b7fff700 1 -- 172.21.15.83:0/2704254131 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49bc011af0 con 0x7f49c0109b90 2024-04-25T11:34:15.541 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.817+0000 7f49c7942700 1 -- 172.21.15.83:0/2704254131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c0109b90 msgr2=0x7f49c0109fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:15.541 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.817+0000 7f49c7942700 1 --2- 172.21.15.83:0/2704254131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c0109b90 0x7f49c0109fb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0098a0 tx=0x7f49bc009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:15.541 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.817+0000 7f49c7942700 1 -- 172.21.15.83:0/2704254131 shutdown_connections 2024-04-25T11:34:15.541 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.817+0000 7f49c7942700 1 --2- 172.21.15.83:0/2704254131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c0109b90 0x7f49c0109fb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:15.541 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.817+0000 7f49c7942700 1 -- 172.21.15.83:0/2704254131 >> 172.21.15.83:0/2704254131 conn(0x7f49c01021d0 msgr2=0x7f49c0104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:15.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.817+0000 7f49c7942700 1 -- 172.21.15.83:0/2704254131 shutdown_connections 2024-04-25T11:34:15.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.817+0000 7f49c7942700 1 -- 172.21.15.83:0/2704254131 wait complete. 2024-04-25T11:34:15.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.817+0000 7f49c7942700 1 Processor -- start 2024-04-25T11:34:15.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.817+0000 7f49c7942700 1 -- start start 2024-04-25T11:34:15.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.818+0000 7f49c7942700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c0109b90 0x7f49c019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:15.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.818+0000 7f49c7942700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49c0107330 con 0x7f49c0109b90 2024-04-25T11:34:15.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.818+0000 7f49c56de700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c0109b90 0x7f49c019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:15.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.818+0000 7f49c56de700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c0109b90 0x7f49c019c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57550/0 (socket says 172.21.15.83:57550) 2024-04-25T11:34:15.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.818+0000 7f49c56de700 1 -- 172.21.15.83:0/3093007703 learned_addr learned my addr 172.21.15.83:0/3093007703 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:15.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.818+0000 7f49c56de700 1 -- 172.21.15.83:0/3093007703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c019ca60 con 0x7f49c0109b90 2024-04-25T11:34:15.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.818+0000 7f49c56de700 1 --2- 172.21.15.83:0/3093007703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c0109b90 0x7f49c019c520 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0117b0 tx=0x7f49bc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:15.542 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.818+0000 7f49b67fc700 1 -- 172.21.15.83:0/3093007703 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49bc003af0 con 0x7f49c0109b90 2024-04-25T11:34:15.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.818+0000 7f49c7942700 1 -- 172.21.15.83:0/3093007703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c019cd50 con 0x7f49c0109b90 2024-04-25T11:34:15.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.818+0000 7f49b67fc700 1 -- 172.21.15.83:0/3093007703 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f49bc020070 con 0x7f49c0109b90 2024-04-25T11:34:15.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.818+0000 7f49b67fc700 1 -- 172.21.15.83:0/3093007703 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49bc019630 con 0x7f49c0109b90 2024-04-25T11:34:15.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.818+0000 7f49c7942700 1 -- 172.21.15.83:0/3093007703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c019f9e0 con 0x7f49c0109b90 2024-04-25T11:34:15.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.819+0000 7f49b67fc700 1 -- 172.21.15.83:0/3093007703 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f49bc0198f0 con 0x7f49c0109b90 2024-04-25T11:34:15.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.819+0000 7f49b67fc700 1 --2- 172.21.15.83:0/3093007703 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f49ac039560 0x7f49ac03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:15.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.819+0000 7f49c7942700 1 -- 172.21.15.83:0/3093007703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c00640a0 con 0x7f49c0109b90 2024-04-25T11:34:15.543 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.819+0000 7f49b67fc700 1 -- 172.21.15.83:0/3093007703 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f49bc04dbc0 con 0x7f49c0109b90 2024-04-25T11:34:15.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.819+0000 7f49c4edd700 1 --2- 172.21.15.83:0/3093007703 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f49ac039560 0x7f49ac03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:15.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.820+0000 7f49c4edd700 1 --2- 172.21.15.83:0/3093007703 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f49ac039560 0x7f49ac03ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f49b0006fd0 tx=0x7f49b0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:15.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:14.824+0000 7f49b67fc700 1 -- 172.21.15.83:0/3093007703 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49bc0042a0 con 0x7f49c0109b90 2024-04-25T11:34:15.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.019+0000 7f49c7942700 1 -- 172.21.15.83:0/3093007703 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f49c019fc10 con 0x7f49c0109b90 2024-04-25T11:34:15.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.020+0000 7f49b67fc700 1 -- 172.21.15.83:0/3093007703 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f49bc04d5e0 con 0x7f49c0109b90 2024-04-25T11:34:15.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.022+0000 7f49c7942700 1 -- 172.21.15.83:0/3093007703 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f49ac039560 msgr2=0x7f49ac03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:15.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.022+0000 7f49c7942700 1 --2- 172.21.15.83:0/3093007703 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f49ac039560 0x7f49ac03ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f49b0006fd0 tx=0x7f49b0006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:15.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.022+0000 7f49c7942700 1 -- 172.21.15.83:0/3093007703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c0109b90 msgr2=0x7f49c019c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:15.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.022+0000 7f49c7942700 1 --2- 172.21.15.83:0/3093007703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c0109b90 0x7f49c019c520 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0117b0 tx=0x7f49bc011b70 comp rx=0 tx=0).stop 2024-04-25T11:34:15.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.022+0000 7f49c7942700 1 -- 172.21.15.83:0/3093007703 shutdown_connections 2024-04-25T11:34:15.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.022+0000 7f49c7942700 1 --2- 172.21.15.83:0/3093007703 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f49ac039560 0x7f49ac03ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:15.544 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.022+0000 7f49c7942700 1 --2- 172.21.15.83:0/3093007703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49c0109b90 0x7f49c019c520 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:15.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.022+0000 7f49c7942700 1 -- 172.21.15.83:0/3093007703 >> 172.21.15.83:0/3093007703 conn(0x7f49c01021d0 msgr2=0x7f49c0102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:15.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.022+0000 7f49c7942700 1 -- 172.21.15.83:0/3093007703 shutdown_connections 2024-04-25T11:34:15.545 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.022+0000 7f49c7942700 1 -- 172.21.15.83:0/3093007703 wait complete. 2024-04-25T11:34:15.545 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling cephadm module... 2024-04-25T11:34:15.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:15 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3093007703' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T11:34:16.484 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:16 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/526590578' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T11:34:16.484 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:16 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: ignoring --setuser ceph since I am not root 2024-04-25T11:34:16.484 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:16 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: ignoring --setgroup ceph since I am not root 2024-04-25T11:34:16.484 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:16 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:16.495+0000 7fc2e40f2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:34:16.773 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:16 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:16.683+0000 7fc2e40f2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:34:16.773 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:16 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:16.784+0000 7fc2e40f2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:34:16.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.949+0000 7f735da33700 1 Processor -- start 2024-04-25T11:34:16.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.949+0000 7f735da33700 1 -- start start 2024-04-25T11:34:16.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.949+0000 7f735da33700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73581074f0 0x7f7358107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:16.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.949+0000 7f735da33700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7358107ee0 con 0x7f73581074f0 2024-04-25T11:34:16.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.949+0000 7f7356ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73581074f0 0x7f7358107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:16.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.949+0000 7f7356ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73581074f0 0x7f7358107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57562/0 (socket says 172.21.15.83:57562) 2024-04-25T11:34:16.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.949+0000 7f7356ffd700 1 -- 172.21.15.83:0/3082452709 learned_addr learned my addr 172.21.15.83:0/3082452709 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:16.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.949+0000 7f7356ffd700 1 -- 172.21.15.83:0/3082452709 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73581086f0 con 0x7f73581074f0 2024-04-25T11:34:16.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.949+0000 7f7356ffd700 1 --2- 172.21.15.83:0/3082452709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73581074f0 0x7f7358107910 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f73480098a0 tx=0x7f7348009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb2a356b53c2f05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:16.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.950+0000 7f7355ffb700 1 -- 172.21.15.83:0/3082452709 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7348004160 con 0x7f73581074f0 2024-04-25T11:34:16.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.950+0000 7f7355ffb700 1 -- 172.21.15.83:0/3082452709 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7348004300 con 0x7f73581074f0 2024-04-25T11:34:16.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.950+0000 7f7355ffb700 1 -- 172.21.15.83:0/3082452709 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7348011af0 con 0x7f73581074f0 2024-04-25T11:34:16.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.950+0000 7f735da33700 1 -- 172.21.15.83:0/3082452709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73581074f0 msgr2=0x7f7358107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:16.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.950+0000 7f735da33700 1 --2- 172.21.15.83:0/3082452709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73581074f0 0x7f7358107910 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f73480098a0 tx=0x7f7348009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:16.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.950+0000 7f735da33700 1 -- 172.21.15.83:0/3082452709 shutdown_connections 2024-04-25T11:34:16.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.950+0000 7f735da33700 1 --2- 172.21.15.83:0/3082452709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73581074f0 0x7f7358107910 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:16.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.950+0000 7f735da33700 1 -- 172.21.15.83:0/3082452709 >> 172.21.15.83:0/3082452709 conn(0x7f7358102a70 msgr2=0x7f7358104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:16.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.950+0000 7f735da33700 1 -- 172.21.15.83:0/3082452709 shutdown_connections 2024-04-25T11:34:16.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.950+0000 7f735da33700 1 -- 172.21.15.83:0/3082452709 wait complete. 2024-04-25T11:34:16.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.951+0000 7f735da33700 1 Processor -- start 2024-04-25T11:34:16.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.951+0000 7f735da33700 1 -- start start 2024-04-25T11:34:16.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.951+0000 7f735da33700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f735819a410 0x7f735819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:16.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.951+0000 7f735da33700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f735807afb0 con 0x7f735819a410 2024-04-25T11:34:16.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.951+0000 7f7356ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f735819a410 0x7f735819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:16.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.951+0000 7f7356ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f735819a410 0x7f735819a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57572/0 (socket says 172.21.15.83:57572) 2024-04-25T11:34:16.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.951+0000 7f7356ffd700 1 -- 172.21.15.83:0/526590578 learned_addr learned my addr 172.21.15.83:0/526590578 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:16.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.952+0000 7f7356ffd700 1 -- 172.21.15.83:0/526590578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735819ad70 con 0x7f735819a410 2024-04-25T11:34:16.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.952+0000 7f7356ffd700 1 --2- 172.21.15.83:0/526590578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f735819a410 0x7f735819a830 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f73480117b0 tx=0x7f7348003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:16.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.952+0000 7f735ca31700 1 -- 172.21.15.83:0/526590578 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73480119c0 con 0x7f735819a410 2024-04-25T11:34:16.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.952+0000 7f735ca31700 1 -- 172.21.15.83:0/526590578 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f734801f070 con 0x7f735819a410 2024-04-25T11:34:16.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.952+0000 7f735da33700 1 -- 172.21.15.83:0/526590578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f735819b000 con 0x7f735819a410 2024-04-25T11:34:16.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.952+0000 7f735ca31700 1 -- 172.21.15.83:0/526590578 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f734801b440 con 0x7f735819a410 2024-04-25T11:34:16.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.952+0000 7f735da33700 1 -- 172.21.15.83:0/526590578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f735819dae0 con 0x7f735819a410 2024-04-25T11:34:16.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.953+0000 7f735ca31700 1 -- 172.21.15.83:0/526590578 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f734801b6b0 con 0x7f735819a410 2024-04-25T11:34:16.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.953+0000 7f735da33700 1 -- 172.21.15.83:0/526590578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73580640a0 con 0x7f735819a410 2024-04-25T11:34:16.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.953+0000 7f735ca31700 1 --2- 172.21.15.83:0/526590578 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f734003b770 0x7f734003dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:16.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.953+0000 7f735ca31700 1 -- 172.21.15.83:0/526590578 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f734804dde0 con 0x7f735819a410 2024-04-25T11:34:16.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.953+0000 7f73567fc700 1 --2- 172.21.15.83:0/526590578 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f734003b770 0x7f734003dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:16.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.953+0000 7f73567fc700 1 --2- 172.21.15.83:0/526590578 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f734003b770 0x7f734003dc30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f734c006fd0 tx=0x7f734c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:16.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:15.958+0000 7f735ca31700 1 -- 172.21.15.83:0/526590578 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f734804d800 con 0x7f735819a410 2024-04-25T11:34:16.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:16.214+0000 7f735da33700 1 -- 172.21.15.83:0/526590578 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f7358193910 con 0x7f735819a410 2024-04-25T11:34:16.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:16.325+0000 7f735ca31700 1 -- 172.21.15.83:0/526590578 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f7358193910 con 0x7f735819a410 2024-04-25T11:34:16.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:16.326+0000 7f735ca31700 1 -- 172.21.15.83:0/526590578 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f734801a7a0 con 0x7f735819a410 2024-04-25T11:34:16.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:16.328+0000 7f735da33700 1 -- 172.21.15.83:0/526590578 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f734003b770 msgr2=0x7f734003dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:16.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:16.328+0000 7f735da33700 1 --2- 172.21.15.83:0/526590578 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f734003b770 0x7f734003dc30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f734c006fd0 tx=0x7f734c006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:16.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:16.328+0000 7f735da33700 1 -- 172.21.15.83:0/526590578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f735819a410 msgr2=0x7f735819a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:16.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:16.328+0000 7f735da33700 1 --2- 172.21.15.83:0/526590578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f735819a410 0x7f735819a830 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f73480117b0 tx=0x7f7348003980 comp rx=0 tx=0).stop 2024-04-25T11:34:16.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:16.328+0000 7f735da33700 1 -- 172.21.15.83:0/526590578 shutdown_connections 2024-04-25T11:34:16.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:16.328+0000 7f735da33700 1 --2- 172.21.15.83:0/526590578 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f734003b770 0x7f734003dc30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:16.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:16.328+0000 7f735da33700 1 --2- 172.21.15.83:0/526590578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f735819a410 0x7f735819a830 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:16.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:16.328+0000 7f735da33700 1 -- 172.21.15.83:0/526590578 >> 172.21.15.83:0/526590578 conn(0x7f7358102a70 msgr2=0x7f73581909d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:16.938 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:16.328+0000 7f735da33700 1 -- 172.21.15.83:0/526590578 shutdown_connections 2024-04-25T11:34:16.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:16.328+0000 7f735da33700 1 -- 172.21.15.83:0/526590578 wait complete. 2024-04-25T11:34:17.314 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:17 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:17.097+0000 7fc2e40f2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:34:17.314 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:17 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:17.181+0000 7fc2e40f2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:34:17.568 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:17 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/526590578' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T11:34:17.569 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:17 smithi083 ceph-mon[29330]: mgrmap e5: x(active, since 6s) 2024-04-25T11:34:18.491 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:18 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1343977626' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T11:34:18.491 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:18 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:18.376+0000 7fc2e40f2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:34:18.500 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:34:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T11:34:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T11:34:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T11:34:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T11:34:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:34:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.563+0000 7fce7f304700 1 Processor -- start 2024-04-25T11:34:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.564+0000 7fce7f304700 1 -- start start 2024-04-25T11:34:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.564+0000 7fce7f304700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce78073ec0 0x7fce780742e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.564+0000 7fce7f304700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fce78074820 con 0x7fce78073ec0 2024-04-25T11:34:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.570+0000 7fce7d0a0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce78073ec0 0x7fce780742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:18.501 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.570+0000 7fce7d0a0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce78073ec0 0x7fce780742e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57606/0 (socket says 172.21.15.83:57606) 2024-04-25T11:34:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.570+0000 7fce7d0a0700 1 -- 172.21.15.83:0/365418460 learned_addr learned my addr 172.21.15.83:0/365418460 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.580+0000 7fce7d0a0700 1 -- 172.21.15.83:0/365418460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce780749a0 con 0x7fce78073ec0 2024-04-25T11:34:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.590+0000 7fce7d0a0700 1 --2- 172.21.15.83:0/365418460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce78073ec0 0x7fce780742e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fce74008920 tx=0x7fce7400d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f276ed76f49e83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.596+0000 7fce6ffff700 1 -- 172.21.15.83:0/365418460 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce7401b070 con 0x7fce78073ec0 2024-04-25T11:34:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.596+0000 7fce6ffff700 1 -- 172.21.15.83:0/365418460 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fce74004030 con 0x7fce78073ec0 2024-04-25T11:34:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.596+0000 7fce6ffff700 1 -- 172.21.15.83:0/365418460 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce74003c00 con 0x7fce78073ec0 2024-04-25T11:34:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.597+0000 7fce7f304700 1 -- 172.21.15.83:0/365418460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce78073ec0 msgr2=0x7fce780742e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.597+0000 7fce7f304700 1 --2- 172.21.15.83:0/365418460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce78073ec0 0x7fce780742e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fce74008920 tx=0x7fce7400d330 comp rx=0 tx=0).stop 2024-04-25T11:34:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.597+0000 7fce7f304700 1 -- 172.21.15.83:0/365418460 shutdown_connections 2024-04-25T11:34:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.597+0000 7fce7f304700 1 --2- 172.21.15.83:0/365418460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce78073ec0 0x7fce780742e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.502 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.597+0000 7fce7f304700 1 -- 172.21.15.83:0/365418460 >> 172.21.15.83:0/365418460 conn(0x7fce7806f2a0 msgr2=0x7fce780716c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:18.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.598+0000 7fce7f304700 1 -- 172.21.15.83:0/365418460 shutdown_connections 2024-04-25T11:34:18.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.600+0000 7fce7f304700 1 -- 172.21.15.83:0/365418460 wait complete. 2024-04-25T11:34:18.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.601+0000 7fce7f304700 1 Processor -- start 2024-04-25T11:34:18.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.601+0000 7fce7f304700 1 -- start start 2024-04-25T11:34:18.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.601+0000 7fce7f304700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce781ab340 0x7fce781ab760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:18.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.601+0000 7fce7f304700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fce78110b20 con 0x7fce781ab340 2024-04-25T11:34:18.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.610+0000 7fce7d0a0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce781ab340 0x7fce781ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:18.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.610+0000 7fce7d0a0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce781ab340 0x7fce781ab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57622/0 (socket says 172.21.15.83:57622) 2024-04-25T11:34:18.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.610+0000 7fce7d0a0700 1 -- 172.21.15.83:0/1343977626 learned_addr learned my addr 172.21.15.83:0/1343977626 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:18.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.610+0000 7fce7d0a0700 1 -- 172.21.15.83:0/1343977626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce781abca0 con 0x7fce781ab340 2024-04-25T11:34:18.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.610+0000 7fce7d0a0700 1 --2- 172.21.15.83:0/1343977626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce781ab340 0x7fce781ab760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fce74003990 tx=0x7fce740099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:18.503 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.611+0000 7fce6e7fc700 1 -- 172.21.15.83:0/1343977626 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce74026070 con 0x7fce781ab340 2024-04-25T11:34:18.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.611+0000 7fce7f304700 1 -- 172.21.15.83:0/1343977626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce781abf30 con 0x7fce781ab340 2024-04-25T11:34:18.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.611+0000 7fce6e7fc700 1 -- 172.21.15.83:0/1343977626 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fce7400b0b0 con 0x7fce781ab340 2024-04-25T11:34:18.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.611+0000 7fce7f304700 1 -- 172.21.15.83:0/1343977626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce781aea10 con 0x7fce781ab340 2024-04-25T11:34:18.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.612+0000 7fce6e7fc700 1 -- 172.21.15.83:0/1343977626 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce74030060 con 0x7fce781ab340 2024-04-25T11:34:18.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.612+0000 7fce63fff700 1 -- 172.21.15.83:0/1343977626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce58005330 con 0x7fce781ab340 2024-04-25T11:34:18.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.615+0000 7fce6e7fc700 1 -- 172.21.15.83:0/1343977626 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fce74029070 con 0x7fce781ab340 2024-04-25T11:34:18.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.615+0000 7fce6e7fc700 1 --2- 172.21.15.83:0/1343977626 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7fce64039560 0x7fce6403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:18.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.615+0000 7fce7c89f700 1 -- 172.21.15.83:0/1343977626 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7fce64039560 msgr2=0x7fce6403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3997740373 2024-04-25T11:34:18.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.615+0000 7fce7c89f700 1 --2- 172.21.15.83:0/1343977626 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7fce64039560 0x7fce6403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:34:18.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.615+0000 7fce6e7fc700 1 -- 172.21.15.83:0/1343977626 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fce74011030 con 0x7fce781ab340 2024-04-25T11:34:18.504 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.621+0000 7fce6e7fc700 1 -- 172.21.15.83:0/1343977626 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce74016440 con 0x7fce781ab340 2024-04-25T11:34:18.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.816+0000 7fce7c89f700 1 -- 172.21.15.83:0/1343977626 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7fce64039560 msgr2=0x7fce6403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3997740373 2024-04-25T11:34:18.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.816+0000 7fce7c89f700 1 --2- 172.21.15.83:0/1343977626 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7fce64039560 0x7fce6403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:34:18.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.884+0000 7fce63fff700 1 -- 172.21.15.83:0/1343977626 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fce580058f0 con 0x7fce781ab340 2024-04-25T11:34:18.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.885+0000 7fce6e7fc700 1 -- 172.21.15.83:0/1343977626 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fce74031ca0 con 0x7fce781ab340 2024-04-25T11:34:18.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.887+0000 7fce63fff700 1 -- 172.21.15.83:0/1343977626 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7fce64039560 msgr2=0x7fce6403ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:34:18.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.887+0000 7fce63fff700 1 --2- 172.21.15.83:0/1343977626 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7fce64039560 0x7fce6403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.887+0000 7fce63fff700 1 -- 172.21.15.83:0/1343977626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce781ab340 msgr2=0x7fce781ab760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:18.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.887+0000 7fce63fff700 1 --2- 172.21.15.83:0/1343977626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce781ab340 0x7fce781ab760 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fce74003990 tx=0x7fce740099f0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.887+0000 7fce63fff700 1 -- 172.21.15.83:0/1343977626 shutdown_connections 2024-04-25T11:34:18.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.887+0000 7fce63fff700 1 --2- 172.21.15.83:0/1343977626 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7fce64039560 0x7fce6403ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.887+0000 7fce63fff700 1 --2- 172.21.15.83:0/1343977626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce781ab340 0x7fce781ab760 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:18.505 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.887+0000 7fce63fff700 1 -- 172.21.15.83:0/1343977626 >> 172.21.15.83:0/1343977626 conn(0x7fce7806f2a0 msgr2=0x7fce78070d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:18.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.887+0000 7fce63fff700 1 -- 172.21.15.83:0/1343977626 shutdown_connections 2024-04-25T11:34:18.506 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:17.887+0000 7fce63fff700 1 -- 172.21.15.83:0/1343977626 wait complete. 2024-04-25T11:34:18.506 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for the mgr to restart... 2024-04-25T11:34:18.506 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr epoch 5... 2024-04-25T11:34:18.827 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:18 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:18.550+0000 7fc2e40f2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:34:18.827 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:18 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:18.719+0000 7fc2e40f2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:34:19.094 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:19 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:19.015+0000 7fc2e40f2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:34:19.366 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:19 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:19.105+0000 7fc2e40f2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:34:19.366 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:19 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:19.315+0000 7fc2e40f2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:34:19.657 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:19 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:19.519+0000 7fc2e40f2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:34:20.407 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:20 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:20.134+0000 7fc2e40f2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:34:20.843 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:20 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:20.560+0000 7fc2e40f2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:34:20.843 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:20 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:20.655+0000 7fc2e40f2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:34:20.843 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:20 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:20.854+0000 7fc2e40f2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:34:21.657 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:21 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:21.385+0000 7fc2e40f2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:34:21.657 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:21 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:21.476+0000 7fc2e40f2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:34:22.046 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:21 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:21.718+0000 7fc2e40f2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:34:22.046 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:21 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:21.849+0000 7fc2e40f2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:34:22.046 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:21 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:21.943+0000 7fc2e40f2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:34:22.046 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:22 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:22.057+0000 7fc2e40f2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:34:22.657 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:22 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:22.360+0000 7fc2e40f2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:34:23.203 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:22 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:22.950+0000 7fc2e40f2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:34:23.204 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:23.162+0000 7fc2e40f2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:34:23.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-mon[29330]: Active manager daemon x restarted 2024-04-25T11:34:23.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-mon[29330]: Activating manager daemon x 2024-04-25T11:34:23.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-mon[29330]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T11:34:23.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-mon[29330]: mgrmap e6: x(active, starting, since 0.00326536s) 2024-04-25T11:34:23.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:34:23.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T11:34:23.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T11:34:23.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T11:34:23.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T11:34:23.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-mon[29330]: Manager daemon x is now available 2024-04-25T11:34:23.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' 2024-04-25T11:34:23.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' 2024-04-25T11:34:23.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:34:23.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:34:23.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:23 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T11:34:24.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:24 smithi083 ceph-mon[29330]: Found migration_current of "None". Setting to last migration. 2024-04-25T11:34:24.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:24 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T11:34:24.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:24 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' 2024-04-25T11:34:24.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:24 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' 2024-04-25T11:34:24.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:24 smithi083 ceph-mon[29330]: mgrmap e7: x(active, since 1.00538s) 2024-04-25T11:34:24.743 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:34:24.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T11:34:24.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T11:34:24.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:34:24.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.938+0000 7f17dcd20700 1 Processor -- start 2024-04-25T11:34:24.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.938+0000 7f17dcd20700 1 -- start start 2024-04-25T11:34:24.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.939+0000 7f17dcd20700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d807de30 0x7f17d807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:24.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.939+0000 7f17dcd20700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17d807c8b0 con 0x7f17d807de30 2024-04-25T11:34:24.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.939+0000 7f17d659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d807de30 0x7f17d807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:24.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.939+0000 7f17d659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d807de30 0x7f17d807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57628/0 (socket says 172.21.15.83:57628) 2024-04-25T11:34:24.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.939+0000 7f17d659c700 1 -- 172.21.15.83:0/2581557635 learned_addr learned my addr 172.21.15.83:0/2581557635 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:24.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.939+0000 7f17d659c700 1 -- 172.21.15.83:0/2581557635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d807ca30 con 0x7f17d807de30 2024-04-25T11:34:24.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.939+0000 7f17d659c700 1 --2- 172.21.15.83:0/2581557635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d807de30 0x7f17d807c2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f17c80098a0 tx=0x7f17c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa602a39956b7414 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:24.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.939+0000 7f17d559a700 1 -- 172.21.15.83:0/2581557635 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c8004160 con 0x7f17d807de30 2024-04-25T11:34:24.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.939+0000 7f17d559a700 1 -- 172.21.15.83:0/2581557635 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f17c8004300 con 0x7f17d807de30 2024-04-25T11:34:24.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.939+0000 7f17d559a700 1 -- 172.21.15.83:0/2581557635 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c8011af0 con 0x7f17d807de30 2024-04-25T11:34:24.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.940+0000 7f17dcd20700 1 -- 172.21.15.83:0/2581557635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d807de30 msgr2=0x7f17d807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:24.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.940+0000 7f17dcd20700 1 --2- 172.21.15.83:0/2581557635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d807de30 0x7f17d807c2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f17c80098a0 tx=0x7f17c8009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:24.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.940+0000 7f17dcd20700 1 -- 172.21.15.83:0/2581557635 shutdown_connections 2024-04-25T11:34:24.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.940+0000 7f17dcd20700 1 --2- 172.21.15.83:0/2581557635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d807de30 0x7f17d807c2e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:24.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.940+0000 7f17dcd20700 1 -- 172.21.15.83:0/2581557635 >> 172.21.15.83:0/2581557635 conn(0x7f17d8103b70 msgr2=0x7f17d8105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:24.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.940+0000 7f17dcd20700 1 -- 172.21.15.83:0/2581557635 shutdown_connections 2024-04-25T11:34:24.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.940+0000 7f17dcd20700 1 -- 172.21.15.83:0/2581557635 wait complete. 2024-04-25T11:34:24.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.941+0000 7f17dcd20700 1 Processor -- start 2024-04-25T11:34:24.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.941+0000 7f17dcd20700 1 -- start start 2024-04-25T11:34:24.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.941+0000 7f17dcd20700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d807de30 0x7f17d819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:24.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.941+0000 7f17dcd20700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17d810eaa0 con 0x7f17d807de30 2024-04-25T11:34:24.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.941+0000 7f17d659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d807de30 0x7f17d819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:24.754 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.941+0000 7f17d659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d807de30 0x7f17d819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:57636/0 (socket says 172.21.15.83:57636) 2024-04-25T11:34:24.754 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.941+0000 7f17d659c700 1 -- 172.21.15.83:0/2106141325 learned_addr learned my addr 172.21.15.83:0/2106141325 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:24.754 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.942+0000 7f17d659c700 1 -- 172.21.15.83:0/2106141325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d819ebb0 con 0x7f17d807de30 2024-04-25T11:34:24.754 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.942+0000 7f17d659c700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d807de30 0x7f17d819e670 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f17c8003f40 tx=0x7f17c8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:24.754 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.942+0000 7f17c77fe700 1 -- 172.21.15.83:0/2106141325 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c80038b0 con 0x7f17d807de30 2024-04-25T11:34:24.754 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.942+0000 7f17dcd20700 1 -- 172.21.15.83:0/2106141325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d819ee40 con 0x7f17d807de30 2024-04-25T11:34:24.754 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.942+0000 7f17c77fe700 1 -- 172.21.15.83:0/2106141325 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f17c8003a50 con 0x7f17d807de30 2024-04-25T11:34:24.754 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.942+0000 7f17c77fe700 1 -- 172.21.15.83:0/2106141325 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c801a5c0 con 0x7f17d807de30 2024-04-25T11:34:24.754 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.942+0000 7f17dcd20700 1 -- 172.21.15.83:0/2106141325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d81a1b30 con 0x7f17d807de30 2024-04-25T11:34:24.754 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.943+0000 7f17c77fe700 1 -- 172.21.15.83:0/2106141325 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f17c801a8c0 con 0x7f17d807de30 2024-04-25T11:34:24.754 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.943+0000 7f17c77fe700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f17c0039560 0x7f17c003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:24.755 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.943+0000 7f17dcd20700 1 -- 172.21.15.83:0/2106141325 --> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f17d8069b80 con 0x7f17c0039560 2024-04-25T11:34:24.755 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.943+0000 7f17c77fe700 1 -- 172.21.15.83:0/2106141325 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f17c804ea20 con 0x7f17d807de30 2024-04-25T11:34:24.755 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.943+0000 7f17d5d9b700 1 -- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f17c0039560 msgr2=0x7f17c003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3997740373 2024-04-25T11:34:24.755 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:18.943+0000 7f17d5d9b700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f17c0039560 0x7f17c003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:34:24.755 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:19.144+0000 7f17d5d9b700 1 -- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f17c0039560 msgr2=0x7f17c003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3997740373 2024-04-25T11:34:24.755 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:19.144+0000 7f17d5d9b700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f17c0039560 0x7f17c003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:34:24.755 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:19.545+0000 7f17d5d9b700 1 -- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f17c0039560 msgr2=0x7f17c003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3997740373 2024-04-25T11:34:24.755 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:19.545+0000 7f17d5d9b700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f17c0039560 0x7f17c003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:34:24.755 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:20.346+0000 7f17d5d9b700 1 -- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f17c0039560 msgr2=0x7f17c003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3997740373 2024-04-25T11:34:24.755 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:20.346+0000 7f17d5d9b700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f17c0039560 0x7f17c003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T11:34:24.755 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:21.947+0000 7f17d5d9b700 1 -- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f17c0039560 msgr2=0x7f17c003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3997740373 2024-04-25T11:34:24.755 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:21.947+0000 7f17d5d9b700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f17c0039560 0x7f17c003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T11:34:24.756 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:23.165+0000 7f17c77fe700 1 -- 172.21.15.83:0/2106141325 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f17c802d910 con 0x7f17d807de30 2024-04-25T11:34:24.756 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:23.165+0000 7f17c77fe700 1 -- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f17c0039560 msgr2=0x7f17c003ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:34:24.756 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:23.165+0000 7f17c77fe700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f17c0039560 0x7f17c003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:24.756 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.166+0000 7f17c77fe700 1 -- 172.21.15.83:0/2106141325 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f17c8017d70 con 0x7f17d807de30 2024-04-25T11:34:24.756 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.166+0000 7f17c77fe700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f17c003cc60 0x7f17c003f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:24.756 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.167+0000 7f17c77fe700 1 -- 172.21.15.83:0/2106141325 --> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f17c8007da0 con 0x7f17c003cc60 2024-04-25T11:34:24.756 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.167+0000 7f17d5d9b700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f17c003cc60 0x7f17c003f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:24.756 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.168+0000 7f17d5d9b700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f17c003cc60 0x7f17c003f050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f17cc003a10 tx=0x7f17cc009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:24.756 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.168+0000 7f17c77fe700 1 -- 172.21.15.83:0/2106141325 <== mgr.14120 v2:172.21.15.83:6800/1639787889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f17c8007da0 con 0x7f17c003cc60 2024-04-25T11:34:24.756 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.174+0000 7f17dcd20700 1 -- 172.21.15.83:0/2106141325 --> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f17d8104e10 con 0x7f17c003cc60 2024-04-25T11:34:24.756 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.175+0000 7f17c77fe700 1 -- 172.21.15.83:0/2106141325 <== mgr.14120 v2:172.21.15.83:6800/1639787889 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f17d8104e10 con 0x7f17c003cc60 2024-04-25T11:34:24.756 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.175+0000 7f17dcd20700 1 -- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f17c003cc60 msgr2=0x7f17c003f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:24.757 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.175+0000 7f17dcd20700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f17c003cc60 0x7f17c003f050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f17cc003a10 tx=0x7f17cc009290 comp rx=0 tx=0).stop 2024-04-25T11:34:24.757 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.175+0000 7f17dcd20700 1 -- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d807de30 msgr2=0x7f17d819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:24.757 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.175+0000 7f17dcd20700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d807de30 0x7f17d819e670 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f17c8003f40 tx=0x7f17c8011890 comp rx=0 tx=0).stop 2024-04-25T11:34:24.757 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.175+0000 7f17dcd20700 1 -- 172.21.15.83:0/2106141325 shutdown_connections 2024-04-25T11:34:24.757 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.175+0000 7f17dcd20700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f17c003cc60 0x7f17c003f050 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:24.757 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.175+0000 7f17dcd20700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d807de30 0x7f17d819e670 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:24.757 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.175+0000 7f17dcd20700 1 --2- 172.21.15.83:0/2106141325 >> [v2:172.21.15.83:6800/3997740373,v1:172.21.15.83:6801/3997740373] conn(0x7f17c0039560 0x7f17c003ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:24.757 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.175+0000 7f17dcd20700 1 -- 172.21.15.83:0/2106141325 >> 172.21.15.83:0/2106141325 conn(0x7f17d8103b70 msgr2=0x7f17d8104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:24.757 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.175+0000 7f17dcd20700 1 -- 172.21.15.83:0/2106141325 shutdown_connections 2024-04-25T11:34:24.757 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:24.175+0000 7f17dcd20700 1 -- 172.21.15.83:0/2106141325 wait complete. 2024-04-25T11:34:24.758 INFO:teuthology.orchestra.run.smithi083.stdout:mgr epoch 5 is available 2024-04-25T11:34:24.758 INFO:teuthology.orchestra.run.smithi083.stdout:Setting orchestrator backend to cephadm... 2024-04-25T11:34:25.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:25 smithi083 ceph-mon[29330]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T11:34:25.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:25 smithi083 ceph-mon[29330]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T11:34:25.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:25 smithi083 ceph-mon[29330]: [25/Apr/2024:11:34:24] ENGINE Bus STARTING 2024-04-25T11:34:25.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:25 smithi083 ceph-mon[29330]: [25/Apr/2024:11:34:24] ENGINE Serving on https://172.21.15.83:7150 2024-04-25T11:34:25.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:25 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:34:25.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.172+0000 7f16e9ece700 1 Processor -- start 2024-04-25T11:34:25.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.172+0000 7f16e9ece700 1 -- start start 2024-04-25T11:34:25.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.172+0000 7f16e9ece700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16e41068b0 0x7f16e4106cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:25.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.172+0000 7f16e9ece700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16e41072a0 con 0x7f16e41068b0 2024-04-25T11:34:25.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.172+0000 7f16e37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16e41068b0 0x7f16e4106cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:25.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.172+0000 7f16e37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16e41068b0 0x7f16e4106cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42120/0 (socket says 172.21.15.83:42120) 2024-04-25T11:34:25.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.172+0000 7f16e37fe700 1 -- 172.21.15.83:0/2579169716 learned_addr learned my addr 172.21.15.83:0/2579169716 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:25.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.172+0000 7f16e37fe700 1 -- 172.21.15.83:0/2579169716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e4109890 con 0x7f16e41068b0 2024-04-25T11:34:25.989 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.173+0000 7f16e37fe700 1 --2- 172.21.15.83:0/2579169716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16e41068b0 0x7f16e4106cd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f16d40098a0 tx=0x7f16d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb4a0e0988d19ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:25.990 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.173+0000 7f16e27fc700 1 -- 172.21.15.83:0/2579169716 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16d4004160 con 0x7f16e41068b0 2024-04-25T11:34:25.990 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.173+0000 7f16e27fc700 1 -- 172.21.15.83:0/2579169716 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f16d4004300 con 0x7f16e41068b0 2024-04-25T11:34:25.990 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.173+0000 7f16e27fc700 1 -- 172.21.15.83:0/2579169716 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16d4011af0 con 0x7f16e41068b0 2024-04-25T11:34:25.990 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.173+0000 7f16e9ece700 1 -- 172.21.15.83:0/2579169716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16e41068b0 msgr2=0x7f16e4106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:25.990 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.173+0000 7f16e9ece700 1 --2- 172.21.15.83:0/2579169716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16e41068b0 0x7f16e4106cd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f16d40098a0 tx=0x7f16d4009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:25.990 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.173+0000 7f16e9ece700 1 -- 172.21.15.83:0/2579169716 shutdown_connections 2024-04-25T11:34:25.990 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.173+0000 7f16e9ece700 1 --2- 172.21.15.83:0/2579169716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16e41068b0 0x7f16e4106cd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:25.990 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.173+0000 7f16e9ece700 1 -- 172.21.15.83:0/2579169716 >> 172.21.15.83:0/2579169716 conn(0x7f16e4102230 msgr2=0x7f16e4104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:25.990 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.173+0000 7f16e9ece700 1 -- 172.21.15.83:0/2579169716 shutdown_connections 2024-04-25T11:34:25.990 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.173+0000 7f16e9ece700 1 -- 172.21.15.83:0/2579169716 wait complete. 2024-04-25T11:34:25.990 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.174+0000 7f16e9ece700 1 Processor -- start 2024-04-25T11:34:25.990 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.174+0000 7f16e9ece700 1 -- start start 2024-04-25T11:34:25.991 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.174+0000 7f16e9ece700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16e41068b0 0x7f16e419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:25.991 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.174+0000 7f16e9ece700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16e410cac0 con 0x7f16e41068b0 2024-04-25T11:34:25.991 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.174+0000 7f16e37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16e41068b0 0x7f16e419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:25.991 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.174+0000 7f16e37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16e41068b0 0x7f16e419c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42128/0 (socket says 172.21.15.83:42128) 2024-04-25T11:34:25.991 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.174+0000 7f16e37fe700 1 -- 172.21.15.83:0/2071495137 learned_addr learned my addr 172.21.15.83:0/2071495137 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:25.991 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.175+0000 7f16e37fe700 1 -- 172.21.15.83:0/2071495137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e419ca50 con 0x7f16e41068b0 2024-04-25T11:34:25.991 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.175+0000 7f16e37fe700 1 --2- 172.21.15.83:0/2071495137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16e41068b0 0x7f16e419c510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f16d4000c00 tx=0x7f16d4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:25.991 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.175+0000 7f16e0ff9700 1 -- 172.21.15.83:0/2071495137 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16d4003b20 con 0x7f16e41068b0 2024-04-25T11:34:25.991 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.175+0000 7f16e0ff9700 1 -- 172.21.15.83:0/2071495137 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f16d401a8c0 con 0x7f16e41068b0 2024-04-25T11:34:25.991 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.175+0000 7f16e0ff9700 1 -- 172.21.15.83:0/2071495137 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16d4024940 con 0x7f16e41068b0 2024-04-25T11:34:25.991 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.175+0000 7f16e9ece700 1 -- 172.21.15.83:0/2071495137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e419cce0 con 0x7f16e41068b0 2024-04-25T11:34:25.991 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.175+0000 7f16e9ece700 1 -- 172.21.15.83:0/2071495137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e419f9d0 con 0x7f16e41068b0 2024-04-25T11:34:25.991 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.176+0000 7f16e0ff9700 1 -- 172.21.15.83:0/2071495137 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f16d4022020 con 0x7f16e41068b0 2024-04-25T11:34:25.992 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.176+0000 7f16e9ece700 1 -- 172.21.15.83:0/2071495137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e40640a0 con 0x7f16e41068b0 2024-04-25T11:34:25.992 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.176+0000 7f16e0ff9700 1 --2- 172.21.15.83:0/2071495137 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f16cc039490 0x7f16cc03b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:25.992 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.176+0000 7f16e0ff9700 1 -- 172.21.15.83:0/2071495137 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f16d404d7e0 con 0x7f16e41068b0 2024-04-25T11:34:25.992 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.176+0000 7f16e2ffd700 1 --2- 172.21.15.83:0/2071495137 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f16cc039490 0x7f16cc03b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:25.992 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.177+0000 7f16e2ffd700 1 --2- 172.21.15.83:0/2071495137 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f16cc039490 0x7f16cc03b950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f16d8006fd0 tx=0x7f16d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:25.992 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.181+0000 7f16e0ff9700 1 -- 172.21.15.83:0/2071495137 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16d402d3f0 con 0x7f16e41068b0 2024-04-25T11:34:25.992 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.218+0000 7f16e0ff9700 1 -- 172.21.15.83:0/2071495137 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f16d4018350 con 0x7f16e41068b0 2024-04-25T11:34:25.992 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.396+0000 7f16e9ece700 1 -- 172.21.15.83:0/2071495137 --> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f16e4103450 con 0x7f16cc039490 2024-04-25T11:34:25.992 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.401+0000 7f16e0ff9700 1 -- 172.21.15.83:0/2071495137 <== mgr.14120 v2:172.21.15.83:6800/1639787889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f16e4103450 con 0x7f16cc039490 2024-04-25T11:34:25.992 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.403+0000 7f16e9ece700 1 -- 172.21.15.83:0/2071495137 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f16cc039490 msgr2=0x7f16cc03b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:25.992 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.403+0000 7f16e9ece700 1 --2- 172.21.15.83:0/2071495137 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f16cc039490 0x7f16cc03b950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f16d8006fd0 tx=0x7f16d8006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:25.992 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.403+0000 7f16e9ece700 1 -- 172.21.15.83:0/2071495137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16e41068b0 msgr2=0x7f16e419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:25.992 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.403+0000 7f16e9ece700 1 --2- 172.21.15.83:0/2071495137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16e41068b0 0x7f16e419c510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f16d4000c00 tx=0x7f16d4011b70 comp rx=0 tx=0).stop 2024-04-25T11:34:25.993 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.404+0000 7f16e9ece700 1 -- 172.21.15.83:0/2071495137 shutdown_connections 2024-04-25T11:34:25.993 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.404+0000 7f16e9ece700 1 --2- 172.21.15.83:0/2071495137 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f16cc039490 0x7f16cc03b950 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:25.993 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.404+0000 7f16e9ece700 1 --2- 172.21.15.83:0/2071495137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16e41068b0 0x7f16e419c510 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:25.993 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.404+0000 7f16e9ece700 1 -- 172.21.15.83:0/2071495137 >> 172.21.15.83:0/2071495137 conn(0x7f16e4102230 msgr2=0x7f16e4102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:25.993 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.404+0000 7f16e9ece700 1 -- 172.21.15.83:0/2071495137 shutdown_connections 2024-04-25T11:34:25.993 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:25.404+0000 7f16e9ece700 1 -- 172.21.15.83:0/2071495137 wait complete. 2024-04-25T11:34:26.315 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:26 smithi083 ceph-mon[29330]: [25/Apr/2024:11:34:24] ENGINE Serving on http://172.21.15.83:8765 2024-04-25T11:34:26.316 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:26 smithi083 ceph-mon[29330]: [25/Apr/2024:11:34:24] ENGINE Bus STARTED 2024-04-25T11:34:26.316 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:26 smithi083 ceph-mon[29330]: mgrmap e8: x(active, since 2s) 2024-04-25T11:34:26.316 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:26 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' 2024-04-25T11:34:26.316 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:26 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:34:27.168 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T11:34:27.168 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.428+0000 7f280ecde700 1 Processor -- start 2024-04-25T11:34:27.168 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.428+0000 7f280ecde700 1 -- start start 2024-04-25T11:34:27.168 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.429+0000 7f280ecde700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28081085d0 0x7f28081089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:27.168 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.429+0000 7f280ecde700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2808108fc0 con 0x7f28081085d0 2024-04-25T11:34:27.168 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.429+0000 7f280ca7a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28081085d0 0x7f28081089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:27.168 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.429+0000 7f280ca7a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28081085d0 0x7f28081089f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42134/0 (socket says 172.21.15.83:42134) 2024-04-25T11:34:27.168 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.429+0000 7f280ca7a700 1 -- 172.21.15.83:0/197704413 learned_addr learned my addr 172.21.15.83:0/197704413 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:27.169 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.429+0000 7f280ca7a700 1 -- 172.21.15.83:0/197704413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28081097d0 con 0x7f28081085d0 2024-04-25T11:34:27.169 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.429+0000 7f280ca7a700 1 --2- 172.21.15.83:0/197704413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28081085d0 0x7f28081089f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f27f80098a0 tx=0x7f27f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8c7444be1cc336 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:27.169 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.429+0000 7f2807fff700 1 -- 172.21.15.83:0/197704413 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f8004160 con 0x7f28081085d0 2024-04-25T11:34:27.169 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.429+0000 7f2807fff700 1 -- 172.21.15.83:0/197704413 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f27f8004300 con 0x7f28081085d0 2024-04-25T11:34:27.169 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.429+0000 7f2807fff700 1 -- 172.21.15.83:0/197704413 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f8011af0 con 0x7f28081085d0 2024-04-25T11:34:27.169 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.430+0000 7f280ecde700 1 -- 172.21.15.83:0/197704413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28081085d0 msgr2=0x7f28081089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:27.169 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.430+0000 7f280ecde700 1 --2- 172.21.15.83:0/197704413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28081085d0 0x7f28081089f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f27f80098a0 tx=0x7f27f8009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:27.169 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.430+0000 7f280ecde700 1 -- 172.21.15.83:0/197704413 shutdown_connections 2024-04-25T11:34:27.169 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.430+0000 7f280ecde700 1 --2- 172.21.15.83:0/197704413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28081085d0 0x7f28081089f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:27.169 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.430+0000 7f280ecde700 1 -- 172.21.15.83:0/197704413 >> 172.21.15.83:0/197704413 conn(0x7f2808103bf0 msgr2=0x7f2808106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:27.169 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.430+0000 7f280ecde700 1 -- 172.21.15.83:0/197704413 shutdown_connections 2024-04-25T11:34:27.169 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.430+0000 7f280ecde700 1 -- 172.21.15.83:0/197704413 wait complete. 2024-04-25T11:34:27.170 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.431+0000 7f280ecde700 1 Processor -- start 2024-04-25T11:34:27.170 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.431+0000 7f280ecde700 1 -- start start 2024-04-25T11:34:27.170 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.431+0000 7f280ecde700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28081085d0 0x7f280807ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:27.170 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.431+0000 7f280ecde700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f280810ca00 con 0x7f28081085d0 2024-04-25T11:34:27.170 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.431+0000 7f280ca7a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28081085d0 0x7f280807ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:27.170 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.431+0000 7f280ca7a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28081085d0 0x7f280807ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42150/0 (socket says 172.21.15.83:42150) 2024-04-25T11:34:27.170 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.431+0000 7f280ca7a700 1 -- 172.21.15.83:0/4045423301 learned_addr learned my addr 172.21.15.83:0/4045423301 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:27.170 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.431+0000 7f280ca7a700 1 -- 172.21.15.83:0/4045423301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280807cf80 con 0x7f28081085d0 2024-04-25T11:34:27.170 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.431+0000 7f280ca7a700 1 --2- 172.21.15.83:0/4045423301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28081085d0 0x7f280807ca40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27f8000c00 tx=0x7f27f8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:27.171 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.431+0000 7f28067fc700 1 -- 172.21.15.83:0/4045423301 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f8003b20 con 0x7f28081085d0 2024-04-25T11:34:27.171 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.431+0000 7f28067fc700 1 -- 172.21.15.83:0/4045423301 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f27f801a8c0 con 0x7f28081085d0 2024-04-25T11:34:27.171 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.431+0000 7f280ecde700 1 -- 172.21.15.83:0/4045423301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280807b170 con 0x7f28081085d0 2024-04-25T11:34:27.171 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.432+0000 7f28067fc700 1 -- 172.21.15.83:0/4045423301 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f8024940 con 0x7f28081085d0 2024-04-25T11:34:27.171 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.432+0000 7f280ecde700 1 -- 172.21.15.83:0/4045423301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280807b6b0 con 0x7f28081085d0 2024-04-25T11:34:27.171 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.432+0000 7f28067fc700 1 -- 172.21.15.83:0/4045423301 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f27f8022020 con 0x7f28081085d0 2024-04-25T11:34:27.171 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.432+0000 7f28067fc700 1 --2- 172.21.15.83:0/4045423301 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f27f4039600 0x7f27f403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:27.171 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.433+0000 7f28067fc700 1 -- 172.21.15.83:0/4045423301 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f27f804d7c0 con 0x7f28081085d0 2024-04-25T11:34:27.171 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.433+0000 7f280ecde700 1 -- 172.21.15.83:0/4045423301 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2808069b80 con 0x7f28081085d0 2024-04-25T11:34:27.171 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.433+0000 7f27fffff700 1 --2- 172.21.15.83:0/4045423301 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f27f4039600 0x7f27f403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:27.171 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.433+0000 7f27fffff700 1 --2- 172.21.15.83:0/4045423301 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f27f4039600 0x7f27f403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f27f0006fd0 tx=0x7f27f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:27.171 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.438+0000 7f28067fc700 1 -- 172.21.15.83:0/4045423301 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27f8021d80 con 0x7f28081085d0 2024-04-25T11:34:27.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.636+0000 7f280ecde700 1 -- 172.21.15.83:0/4045423301 --> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f280807c000 con 0x7f27f4039600 2024-04-25T11:34:27.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.637+0000 7f28067fc700 1 -- 172.21.15.83:0/4045423301 <== mgr.14120 v2:172.21.15.83:6800/1639787889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f280807c000 con 0x7f27f4039600 2024-04-25T11:34:27.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.639+0000 7f280ecde700 1 -- 172.21.15.83:0/4045423301 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f27f4039600 msgr2=0x7f27f403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:27.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.639+0000 7f280ecde700 1 --2- 172.21.15.83:0/4045423301 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f27f4039600 0x7f27f403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f27f0006fd0 tx=0x7f27f0006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:27.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.639+0000 7f280ecde700 1 -- 172.21.15.83:0/4045423301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28081085d0 msgr2=0x7f280807ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:27.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.639+0000 7f280ecde700 1 --2- 172.21.15.83:0/4045423301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28081085d0 0x7f280807ca40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27f8000c00 tx=0x7f27f8011b70 comp rx=0 tx=0).stop 2024-04-25T11:34:27.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.639+0000 7f280ecde700 1 -- 172.21.15.83:0/4045423301 shutdown_connections 2024-04-25T11:34:27.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.639+0000 7f280ecde700 1 --2- 172.21.15.83:0/4045423301 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f27f4039600 0x7f27f403bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:27.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.639+0000 7f280ecde700 1 --2- 172.21.15.83:0/4045423301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28081085d0 0x7f280807ca40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:27.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.639+0000 7f280ecde700 1 -- 172.21.15.83:0/4045423301 >> 172.21.15.83:0/4045423301 conn(0x7f2808103bf0 msgr2=0x7f2808104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:27.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.639+0000 7f280ecde700 1 -- 172.21.15.83:0/4045423301 shutdown_connections 2024-04-25T11:34:27.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:26.639+0000 7f280ecde700 1 -- 172.21.15.83:0/4045423301 wait complete. 2024-04-25T11:34:27.173 INFO:teuthology.orchestra.run.smithi083.stdout:Generating ssh key... 2024-04-25T11:34:27.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-mon[29330]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:34:28.157 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: Generating public/private rsa key pair. 2024-04-25T11:34:28.157 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: Your identification has been saved in /tmp/tmpll8x47n5/key. 2024-04-25T11:34:28.157 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: Your public key has been saved in /tmp/tmpll8x47n5/key.pub. 2024-04-25T11:34:28.158 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: The key fingerprint is: 2024-04-25T11:34:28.158 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: SHA256:/lz7VG8PcXnKl3dQJqgqWpPHtzJ4Rbn+w33ZJ5HHBT0 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:34:28.158 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: The key's randomart image is: 2024-04-25T11:34:28.158 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: +---[RSA 3072]----+ 2024-04-25T11:34:28.158 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: | . | 2024-04-25T11:34:28.158 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: | ..E.| 2024-04-25T11:34:28.158 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: | .. ..+| 2024-04-25T11:34:28.158 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: | o. +o| 2024-04-25T11:34:28.158 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: | S... o+=| 2024-04-25T11:34:28.158 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: | + .o .o**| 2024-04-25T11:34:28.158 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: | =.=o....=+O| 2024-04-25T11:34:28.158 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: | o.+++.oooo=B| 2024-04-25T11:34:28.158 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: | . . o+.oo.oo| 2024-04-25T11:34:28.158 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: +----[SHA256]-----+ 2024-04-25T11:34:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.635+0000 7fbe978ce700 1 Processor -- start 2024-04-25T11:34:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.635+0000 7fbe978ce700 1 -- start start 2024-04-25T11:34:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.635+0000 7fbe978ce700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90108ad0 0x7fbe90108ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.635+0000 7fbe978ce700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbe901094c0 con 0x7fbe90108ad0 2024-04-25T11:34:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.636+0000 7fbe9566a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90108ad0 0x7fbe90108ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:28.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.636+0000 7fbe9566a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90108ad0 0x7fbe90108ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42156/0 (socket says 172.21.15.83:42156) 2024-04-25T11:34:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.636+0000 7fbe9566a700 1 -- 172.21.15.83:0/1800896332 learned_addr learned my addr 172.21.15.83:0/1800896332 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.636+0000 7fbe9566a700 1 -- 172.21.15.83:0/1800896332 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe90107780 con 0x7fbe90108ad0 2024-04-25T11:34:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.636+0000 7fbe9566a700 1 --2- 172.21.15.83:0/1800896332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90108ad0 0x7fbe90108ef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009e80 tx=0x7fbe8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12a3299ee1a837f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.636+0000 7fbe87fff700 1 -- 172.21.15.83:0/1800896332 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe8c017070 con 0x7fbe90108ad0 2024-04-25T11:34:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.636+0000 7fbe87fff700 1 -- 172.21.15.83:0/1800896332 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbe8c00b8c0 con 0x7fbe90108ad0 2024-04-25T11:34:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.636+0000 7fbe87fff700 1 -- 172.21.15.83:0/1800896332 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe8c004490 con 0x7fbe90108ad0 2024-04-25T11:34:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.637+0000 7fbe978ce700 1 -- 172.21.15.83:0/1800896332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90108ad0 msgr2=0x7fbe90108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.637+0000 7fbe978ce700 1 --2- 172.21.15.83:0/1800896332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90108ad0 0x7fbe90108ef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009e80 tx=0x7fbe8c00b330 comp rx=0 tx=0).stop 2024-04-25T11:34:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.637+0000 7fbe978ce700 1 -- 172.21.15.83:0/1800896332 shutdown_connections 2024-04-25T11:34:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.637+0000 7fbe978ce700 1 --2- 172.21.15.83:0/1800896332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90108ad0 0x7fbe90108ef0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.637+0000 7fbe978ce700 1 -- 172.21.15.83:0/1800896332 >> 172.21.15.83:0/1800896332 conn(0x7fbe90102230 msgr2=0x7fbe90104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:28.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.637+0000 7fbe978ce700 1 -- 172.21.15.83:0/1800896332 shutdown_connections 2024-04-25T11:34:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.637+0000 7fbe978ce700 1 -- 172.21.15.83:0/1800896332 wait complete. 2024-04-25T11:34:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.637+0000 7fbe978ce700 1 Processor -- start 2024-04-25T11:34:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.637+0000 7fbe978ce700 1 -- start start 2024-04-25T11:34:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.637+0000 7fbe978ce700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90108ad0 0x7fbe901070f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.637+0000 7fbe978ce700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9010b9c0 con 0x7fbe90108ad0 2024-04-25T11:34:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.638+0000 7fbe9566a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90108ad0 0x7fbe901070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.638+0000 7fbe9566a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90108ad0 0x7fbe901070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42172/0 (socket says 172.21.15.83:42172) 2024-04-25T11:34:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.638+0000 7fbe9566a700 1 -- 172.21.15.83:0/864763573 learned_addr learned my addr 172.21.15.83:0/864763573 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.638+0000 7fbe9566a700 1 -- 172.21.15.83:0/864763573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9019f210 con 0x7fbe90108ad0 2024-04-25T11:34:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.638+0000 7fbe9566a700 1 --2- 172.21.15.83:0/864763573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90108ad0 0x7fbe901070f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c004a60 tx=0x7fbe8c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.638+0000 7fbe867fc700 1 -- 172.21.15.83:0/864763573 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe8c020020 con 0x7fbe90108ad0 2024-04-25T11:34:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.638+0000 7fbe867fc700 1 -- 172.21.15.83:0/864763573 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbe8c003c60 con 0x7fbe90108ad0 2024-04-25T11:34:28.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.638+0000 7fbe978ce700 1 -- 172.21.15.83:0/864763573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9019f3e0 con 0x7fbe90108ad0 2024-04-25T11:34:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.638+0000 7fbe867fc700 1 -- 172.21.15.83:0/864763573 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe8c017040 con 0x7fbe90108ad0 2024-04-25T11:34:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.638+0000 7fbe978ce700 1 -- 172.21.15.83:0/864763573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9019f800 con 0x7fbe90108ad0 2024-04-25T11:34:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.639+0000 7fbe867fc700 1 -- 172.21.15.83:0/864763573 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fbe8c023020 con 0x7fbe90108ad0 2024-04-25T11:34:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.639+0000 7fbe867fc700 1 --2- 172.21.15.83:0/864763573 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fbe7c039600 0x7fbe7c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.639+0000 7fbe867fc700 1 -- 172.21.15.83:0/864763573 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbe8c00f050 con 0x7fbe90108ad0 2024-04-25T11:34:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.639+0000 7fbe978ce700 1 -- 172.21.15.83:0/864763573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe900640a0 con 0x7fbe90108ad0 2024-04-25T11:34:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.639+0000 7fbe94e69700 1 --2- 172.21.15.83:0/864763573 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fbe7c039600 0x7fbe7c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.640+0000 7fbe94e69700 1 --2- 172.21.15.83:0/864763573 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fbe7c039600 0x7fbe7c03bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe80006fd0 tx=0x7fbe80006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.644+0000 7fbe867fc700 1 -- 172.21.15.83:0/864763573 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe8c02a020 con 0x7fbe90108ad0 2024-04-25T11:34:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.841+0000 7fbe978ce700 1 -- 172.21.15.83:0/864763573 --> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fbe90103400 con 0x7fbe7c039600 2024-04-25T11:34:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.974+0000 7fbe867fc700 1 -- 172.21.15.83:0/864763573 <== mgr.14120 v2:172.21.15.83:6800/1639787889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbe90103400 con 0x7fbe7c039600 2024-04-25T11:34:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.976+0000 7fbe978ce700 1 -- 172.21.15.83:0/864763573 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fbe7c039600 msgr2=0x7fbe7c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.976+0000 7fbe978ce700 1 --2- 172.21.15.83:0/864763573 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fbe7c039600 0x7fbe7c03bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe80006fd0 tx=0x7fbe80006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:28.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.976+0000 7fbe978ce700 1 -- 172.21.15.83:0/864763573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90108ad0 msgr2=0x7fbe901070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.976+0000 7fbe978ce700 1 --2- 172.21.15.83:0/864763573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90108ad0 0x7fbe901070f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c004a60 tx=0x7fbe8c00bd50 comp rx=0 tx=0).stop 2024-04-25T11:34:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.976+0000 7fbe978ce700 1 -- 172.21.15.83:0/864763573 shutdown_connections 2024-04-25T11:34:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.976+0000 7fbe978ce700 1 --2- 172.21.15.83:0/864763573 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fbe7c039600 0x7fbe7c03bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.976+0000 7fbe978ce700 1 --2- 172.21.15.83:0/864763573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe90108ad0 0x7fbe901070f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.976+0000 7fbe978ce700 1 -- 172.21.15.83:0/864763573 >> 172.21.15.83:0/864763573 conn(0x7fbe90102230 msgr2=0x7fbe90102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.976+0000 7fbe978ce700 1 -- 172.21.15.83:0/864763573 shutdown_connections 2024-04-25T11:34:28.454 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:27.976+0000 7fbe978ce700 1 -- 172.21.15.83:0/864763573 wait complete. 2024-04-25T11:34:28.551 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:28 smithi083 ceph-mon[29330]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:34:28.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:28 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' 2024-04-25T11:34:28.552 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:28 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' 2024-04-25T11:34:29.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:29 smithi083 ceph-mon[29330]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:34:29.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:29 smithi083 ceph-mon[29330]: Generating ssh key... 2024-04-25T11:34:29.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDW0qtjjPz5Gr7fr59HtylgjXV9iF2/G/BPc/I5bYfu2pn7SaxS4Uav/cVSGfZ+jT0nwUpsUNQSqVdjrWHMubRs1J6xTDMBxSE+n4DFVoWFldsjsnnrfGUr42KLlndTr8tOY/zEkF0xhTAATa2PWL61wn5p5R/PtdcdrBB9a3fh92nhTlwz7RKwKdEDt8w0wacT/oHn3ctqJfZipN3ttHzZm8+0p6nWsgKKzrrw7w2Fj/XL6UKOl3bgodyEIp4jLf1IFSqudHBZaxPpCCiyXwK/5ulRz23R881seyYCxWTE2An4C6gK/UkF12sbXtpjec8wUwjXurq3p4TKBUNEUYl9IQd3p3+/uovr9bLEGud0OdjbwURERtt5L3rUY7v93PBRi+7cR91wh+t2JJkgM4PvBE7x8ihSl+VRHuvcbi647zXSFy4tJrGWe2Xi7Hjm85hfvuTcG0jo1UI0KFGL+aAE3q+Z9UZ0d6xrRLpsIaWzJMQx1bYqTprf13BtRqJCOM0= ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:34:29.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.848+0000 7f7ce66c8700 1 Processor -- start 2024-04-25T11:34:29.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.848+0000 7f7ce66c8700 1 -- start start 2024-04-25T11:34:29.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.848+0000 7f7ce66c8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce01074d0 0x7f7ce01078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:29.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.848+0000 7f7ce66c8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce0107ec0 con 0x7f7ce01074d0 2024-04-25T11:34:29.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.849+0000 7f7cdffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce01074d0 0x7f7ce01078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:29.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.849+0000 7f7cdffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce01074d0 0x7f7ce01078f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42186/0 (socket says 172.21.15.83:42186) 2024-04-25T11:34:29.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.849+0000 7f7cdffff700 1 -- 172.21.15.83:0/2334533276 learned_addr learned my addr 172.21.15.83:0/2334533276 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:29.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.849+0000 7f7cdffff700 1 -- 172.21.15.83:0/2334533276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce01086d0 con 0x7f7ce01074d0 2024-04-25T11:34:29.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.849+0000 7f7cdffff700 1 --2- 172.21.15.83:0/2334533276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce01074d0 0x7f7ce01078f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000bb40 tx=0x7f7cd000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f79ff568724554 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:29.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.849+0000 7f7cdeffd700 1 -- 172.21.15.83:0/2334533276 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cd000dbe0 con 0x7f7ce01074d0 2024-04-25T11:34:29.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.849+0000 7f7cdeffd700 1 -- 172.21.15.83:0/2334533276 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7cd00044c0 con 0x7f7ce01074d0 2024-04-25T11:34:29.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.849+0000 7f7cdeffd700 1 -- 172.21.15.83:0/2334533276 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cd0010470 con 0x7f7ce01074d0 2024-04-25T11:34:29.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.850+0000 7f7ce66c8700 1 -- 172.21.15.83:0/2334533276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce01074d0 msgr2=0x7f7ce01078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:29.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.850+0000 7f7ce66c8700 1 --2- 172.21.15.83:0/2334533276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce01074d0 0x7f7ce01078f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000bb40 tx=0x7f7cd000be50 comp rx=0 tx=0).stop 2024-04-25T11:34:29.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.850+0000 7f7ce66c8700 1 -- 172.21.15.83:0/2334533276 shutdown_connections 2024-04-25T11:34:29.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.850+0000 7f7ce66c8700 1 --2- 172.21.15.83:0/2334533276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce01074d0 0x7f7ce01078f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:29.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.850+0000 7f7ce66c8700 1 -- 172.21.15.83:0/2334533276 >> 172.21.15.83:0/2334533276 conn(0x7f7ce0102ab0 msgr2=0x7f7ce0104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:29.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.850+0000 7f7ce66c8700 1 -- 172.21.15.83:0/2334533276 shutdown_connections 2024-04-25T11:34:29.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.850+0000 7f7ce66c8700 1 -- 172.21.15.83:0/2334533276 wait complete. 2024-04-25T11:34:29.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.850+0000 7f7ce66c8700 1 Processor -- start 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.850+0000 7f7ce66c8700 1 -- start start 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.851+0000 7f7ce66c8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce01074d0 0x7f7ce019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.851+0000 7f7ce66c8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce0062e80 con 0x7f7ce01074d0 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.851+0000 7f7cdffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce01074d0 0x7f7ce019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.851+0000 7f7cdffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce01074d0 0x7f7ce019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42200/0 (socket says 172.21.15.83:42200) 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.851+0000 7f7cdffff700 1 -- 172.21.15.83:0/1354961700 learned_addr learned my addr 172.21.15.83:0/1354961700 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.851+0000 7f7cdffff700 1 -- 172.21.15.83:0/1354961700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce019a790 con 0x7f7ce01074d0 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.851+0000 7f7cdffff700 1 --2- 172.21.15.83:0/1354961700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce01074d0 0x7f7ce019a250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000b340 tx=0x7f7cd0003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.851+0000 7f7cdd7fa700 1 -- 172.21.15.83:0/1354961700 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cd0027070 con 0x7f7ce01074d0 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.851+0000 7f7cdd7fa700 1 -- 172.21.15.83:0/1354961700 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7cd0010bb0 con 0x7f7ce01074d0 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.851+0000 7f7ce66c8700 1 -- 172.21.15.83:0/1354961700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce019aa20 con 0x7f7ce01074d0 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.851+0000 7f7cdd7fa700 1 -- 172.21.15.83:0/1354961700 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cd0022ac0 con 0x7f7ce01074d0 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.851+0000 7f7ce66c8700 1 -- 172.21.15.83:0/1354961700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce019d710 con 0x7f7ce01074d0 2024-04-25T11:34:29.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.852+0000 7f7cdd7fa700 1 -- 172.21.15.83:0/1354961700 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7cd002a070 con 0x7f7ce01074d0 2024-04-25T11:34:29.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.852+0000 7f7ce66c8700 1 -- 172.21.15.83:0/1354961700 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce00640a0 con 0x7f7ce01074d0 2024-04-25T11:34:29.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.852+0000 7f7cdd7fa700 1 --2- 172.21.15.83:0/1354961700 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f7cc8039600 0x7f7cc803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:29.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.852+0000 7f7cdd7fa700 1 -- 172.21.15.83:0/1354961700 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7cd001f070 con 0x7f7ce01074d0 2024-04-25T11:34:29.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.852+0000 7f7cdf7fe700 1 --2- 172.21.15.83:0/1354961700 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f7cc8039600 0x7f7cc803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:29.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.853+0000 7f7cdf7fe700 1 --2- 172.21.15.83:0/1354961700 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f7cc8039600 0x7f7cc803bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4006fd0 tx=0x7f7cd4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:29.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:28.857+0000 7f7cdd7fa700 1 -- 172.21.15.83:0/1354961700 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7cd0015070 con 0x7f7ce01074d0 2024-04-25T11:34:29.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:29.055+0000 7f7ce66c8700 1 -- 172.21.15.83:0/1354961700 --> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f7ce00753f0 con 0x7f7cc8039600 2024-04-25T11:34:29.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:29.056+0000 7f7cdd7fa700 1 -- 172.21.15.83:0/1354961700 <== mgr.14120 v2:172.21.15.83:6800/1639787889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f7ce00753f0 con 0x7f7cc8039600 2024-04-25T11:34:29.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:29.058+0000 7f7ce66c8700 1 -- 172.21.15.83:0/1354961700 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f7cc8039600 msgr2=0x7f7cc803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:29.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:29.058+0000 7f7ce66c8700 1 --2- 172.21.15.83:0/1354961700 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f7cc8039600 0x7f7cc803bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4006fd0 tx=0x7f7cd4006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:29.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:29.058+0000 7f7ce66c8700 1 -- 172.21.15.83:0/1354961700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce01074d0 msgr2=0x7f7ce019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:29.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:29.058+0000 7f7ce66c8700 1 --2- 172.21.15.83:0/1354961700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce01074d0 0x7f7ce019a250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000b340 tx=0x7f7cd0003dc0 comp rx=0 tx=0).stop 2024-04-25T11:34:29.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:29.058+0000 7f7ce66c8700 1 -- 172.21.15.83:0/1354961700 shutdown_connections 2024-04-25T11:34:29.669 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:29.058+0000 7f7ce66c8700 1 --2- 172.21.15.83:0/1354961700 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f7cc8039600 0x7f7cc803bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:29.669 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:29.058+0000 7f7ce66c8700 1 --2- 172.21.15.83:0/1354961700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ce01074d0 0x7f7ce019a250 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:29.669 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:29.058+0000 7f7ce66c8700 1 -- 172.21.15.83:0/1354961700 >> 172.21.15.83:0/1354961700 conn(0x7f7ce0102ab0 msgr2=0x7f7ce0190890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:29.669 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:29.058+0000 7f7ce66c8700 1 -- 172.21.15.83:0/1354961700 shutdown_connections 2024-04-25T11:34:29.669 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:29.058+0000 7f7ce66c8700 1 -- 172.21.15.83:0/1354961700 wait complete. 2024-04-25T11:34:29.669 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T11:34:29.669 INFO:teuthology.orchestra.run.smithi083.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T11:34:29.669 INFO:teuthology.orchestra.run.smithi083.stdout:Adding host smithi083... 2024-04-25T11:34:30.484 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:30 smithi083 ceph-mon[29330]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:34:31.313 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:31 smithi083 ceph-mon[29330]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi083", "addr": "172.21.15.83", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:34:32.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:32 smithi083 ceph-mon[29330]: Deploying cephadm binary to smithi083 2024-04-25T11:34:34.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Added host 'smithi083' with addr '172.21.15.83' 2024-04-25T11:34:34.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.128+0000 7f2315ed8700 1 Processor -- start 2024-04-25T11:34:34.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.128+0000 7f2315ed8700 1 -- start start 2024-04-25T11:34:34.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.128+0000 7f2315ed8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23101074d0 0x7f23101078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:34.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.128+0000 7f2315ed8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2310107ec0 con 0x7f23101074d0 2024-04-25T11:34:34.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.129+0000 7f230f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23101074d0 0x7f23101078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:34.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.129+0000 7f230f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23101074d0 0x7f23101078f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42210/0 (socket says 172.21.15.83:42210) 2024-04-25T11:34:34.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.129+0000 7f230f7fe700 1 -- 172.21.15.83:0/406191689 learned_addr learned my addr 172.21.15.83:0/406191689 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:34.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.129+0000 7f230f7fe700 1 -- 172.21.15.83:0/406191689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23101086d0 con 0x7f23101074d0 2024-04-25T11:34:34.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.129+0000 7f230f7fe700 1 --2- 172.21.15.83:0/406191689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23101074d0 0x7f23101078f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f23000098a0 tx=0x7f2300009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7533cd6d6350882 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:34.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.129+0000 7f230e7fc700 1 -- 172.21.15.83:0/406191689 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2300004160 con 0x7f23101074d0 2024-04-25T11:34:34.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.129+0000 7f230e7fc700 1 -- 172.21.15.83:0/406191689 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2300004300 con 0x7f23101074d0 2024-04-25T11:34:34.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.129+0000 7f230e7fc700 1 -- 172.21.15.83:0/406191689 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2300011af0 con 0x7f23101074d0 2024-04-25T11:34:34.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.130+0000 7f2315ed8700 1 -- 172.21.15.83:0/406191689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23101074d0 msgr2=0x7f23101078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:34.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.130+0000 7f2315ed8700 1 --2- 172.21.15.83:0/406191689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23101074d0 0x7f23101078f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f23000098a0 tx=0x7f2300009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:34.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.130+0000 7f2315ed8700 1 -- 172.21.15.83:0/406191689 shutdown_connections 2024-04-25T11:34:34.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.130+0000 7f2315ed8700 1 --2- 172.21.15.83:0/406191689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23101074d0 0x7f23101078f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:34.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.130+0000 7f2315ed8700 1 -- 172.21.15.83:0/406191689 >> 172.21.15.83:0/406191689 conn(0x7f2310102ab0 msgr2=0x7f2310104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:34.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.130+0000 7f2315ed8700 1 -- 172.21.15.83:0/406191689 shutdown_connections 2024-04-25T11:34:34.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.130+0000 7f2315ed8700 1 -- 172.21.15.83:0/406191689 wait complete. 2024-04-25T11:34:34.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.130+0000 7f2315ed8700 1 Processor -- start 2024-04-25T11:34:34.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.131+0000 7f2315ed8700 1 -- start start 2024-04-25T11:34:34.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.131+0000 7f2315ed8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23101074d0 0x7f231019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:34.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.131+0000 7f2315ed8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f231010b900 con 0x7f23101074d0 2024-04-25T11:34:34.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.131+0000 7f230f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23101074d0 0x7f231019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:34.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.131+0000 7f230f7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23101074d0 0x7f231019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42224/0 (socket says 172.21.15.83:42224) 2024-04-25T11:34:34.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.131+0000 7f230f7fe700 1 -- 172.21.15.83:0/513982689 learned_addr learned my addr 172.21.15.83:0/513982689 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:34.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.131+0000 7f230f7fe700 1 -- 172.21.15.83:0/513982689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231019ca90 con 0x7f23101074d0 2024-04-25T11:34:34.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.131+0000 7f230f7fe700 1 --2- 172.21.15.83:0/513982689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23101074d0 0x7f231019c550 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2300003f40 tx=0x7f2300011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:34.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.131+0000 7f230cff9700 1 -- 172.21.15.83:0/513982689 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23000038b0 con 0x7f23101074d0 2024-04-25T11:34:34.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.131+0000 7f230cff9700 1 -- 172.21.15.83:0/513982689 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2300003a50 con 0x7f23101074d0 2024-04-25T11:34:34.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.131+0000 7f2315ed8700 1 -- 172.21.15.83:0/513982689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231019cd20 con 0x7f23101074d0 2024-04-25T11:34:34.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.131+0000 7f230cff9700 1 -- 172.21.15.83:0/513982689 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f230001a5c0 con 0x7f23101074d0 2024-04-25T11:34:34.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.131+0000 7f2315ed8700 1 -- 172.21.15.83:0/513982689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f231019f9c0 con 0x7f23101074d0 2024-04-25T11:34:34.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.132+0000 7f230cff9700 1 -- 172.21.15.83:0/513982689 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f230001a8c0 con 0x7f23101074d0 2024-04-25T11:34:34.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.133+0000 7f2315ed8700 1 -- 172.21.15.83:0/513982689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f231010ba80 con 0x7f23101074d0 2024-04-25T11:34:34.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.133+0000 7f230cff9700 1 --2- 172.21.15.83:0/513982689 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f22f8039560 0x7f22f803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:34.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.133+0000 7f230cff9700 1 -- 172.21.15.83:0/513982689 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f230004dc70 con 0x7f23101074d0 2024-04-25T11:34:34.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.133+0000 7f230effd700 1 --2- 172.21.15.83:0/513982689 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f22f8039560 0x7f22f803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:34.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.133+0000 7f230effd700 1 --2- 172.21.15.83:0/513982689 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f22f8039560 0x7f22f803ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2304006fd0 tx=0x7f2304006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:34.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.138+0000 7f230cff9700 1 -- 172.21.15.83:0/513982689 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2300019c70 con 0x7f23101074d0 2024-04-25T11:34:34.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:30.344+0000 7f2315ed8700 1 -- 172.21.15.83:0/513982689 --> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi083", "addr": "172.21.15.83", "target": ["mon-mgr", ""]}) -- 0x7f231002e260 con 0x7f22f8039560 2024-04-25T11:34:34.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:34.370+0000 7f230cff9700 1 -- 172.21.15.83:0/513982689 <== mgr.14120 v2:172.21.15.83:6800/1639787889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f231002e260 con 0x7f22f8039560 2024-04-25T11:34:34.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:34.373+0000 7f2315ed8700 1 -- 172.21.15.83:0/513982689 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f22f8039560 msgr2=0x7f22f803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:34.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:34.373+0000 7f2315ed8700 1 --2- 172.21.15.83:0/513982689 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f22f8039560 0x7f22f803ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2304006fd0 tx=0x7f2304006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:34.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:34.373+0000 7f2315ed8700 1 -- 172.21.15.83:0/513982689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23101074d0 msgr2=0x7f231019c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:34.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:34.373+0000 7f2315ed8700 1 --2- 172.21.15.83:0/513982689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23101074d0 0x7f231019c550 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2300003f40 tx=0x7f2300011890 comp rx=0 tx=0).stop 2024-04-25T11:34:34.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:34.373+0000 7f2315ed8700 1 -- 172.21.15.83:0/513982689 shutdown_connections 2024-04-25T11:34:34.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:34.373+0000 7f2315ed8700 1 --2- 172.21.15.83:0/513982689 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f22f8039560 0x7f22f803ba20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:34.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:34.373+0000 7f2315ed8700 1 --2- 172.21.15.83:0/513982689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23101074d0 0x7f231019c550 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:34.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:34.373+0000 7f2315ed8700 1 -- 172.21.15.83:0/513982689 >> 172.21.15.83:0/513982689 conn(0x7f2310102ab0 msgr2=0x7f2310103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:34.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:34.373+0000 7f2315ed8700 1 -- 172.21.15.83:0/513982689 shutdown_connections 2024-04-25T11:34:34.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:34.373+0000 7f2315ed8700 1 -- 172.21.15.83:0/513982689 wait complete. 2024-04-25T11:34:34.936 INFO:teuthology.orchestra.run.smithi083.stdout:Deploying unmanaged mon service... 2024-04-25T11:34:35.583 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:35 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' 2024-04-25T11:34:35.583 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:35 smithi083 ceph-mon[29330]: Added host smithi083 2024-04-25T11:34:35.584 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:35 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:34:36.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T11:34:36.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.362+0000 7faca7de2700 1 Processor -- start 2024-04-25T11:34:36.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.363+0000 7faca7de2700 1 -- start start 2024-04-25T11:34:36.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.363+0000 7faca7de2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca0106e80 0x7faca01072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:36.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.363+0000 7faca7de2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faca007be30 con 0x7faca0106e80 2024-04-25T11:34:36.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.363+0000 7faca5b7e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca0106e80 0x7faca01072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:36.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.363+0000 7faca5b7e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca0106e80 0x7faca01072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53342/0 (socket says 172.21.15.83:53342) 2024-04-25T11:34:36.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.363+0000 7faca5b7e700 1 -- 172.21.15.83:0/2651759462 learned_addr learned my addr 172.21.15.83:0/2651759462 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:36.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.363+0000 7faca5b7e700 1 -- 172.21.15.83:0/2651759462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca01077e0 con 0x7faca0106e80 2024-04-25T11:34:36.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.364+0000 7faca5b7e700 1 --2- 172.21.15.83:0/2651759462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca0106e80 0x7faca01072a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fac98009e80 tx=0x7fac9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e880ef6d346a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:36.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.364+0000 7faca4b7c700 1 -- 172.21.15.83:0/2651759462 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac98017070 con 0x7faca0106e80 2024-04-25T11:34:36.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.364+0000 7faca4b7c700 1 -- 172.21.15.83:0/2651759462 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac9800b8c0 con 0x7faca0106e80 2024-04-25T11:34:36.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.364+0000 7faca4b7c700 1 -- 172.21.15.83:0/2651759462 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac98004490 con 0x7faca0106e80 2024-04-25T11:34:36.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.364+0000 7faca7de2700 1 -- 172.21.15.83:0/2651759462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca0106e80 msgr2=0x7faca01072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:36.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.364+0000 7faca7de2700 1 --2- 172.21.15.83:0/2651759462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca0106e80 0x7faca01072a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fac98009e80 tx=0x7fac9800b330 comp rx=0 tx=0).stop 2024-04-25T11:34:36.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.364+0000 7faca7de2700 1 -- 172.21.15.83:0/2651759462 shutdown_connections 2024-04-25T11:34:36.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.364+0000 7faca7de2700 1 --2- 172.21.15.83:0/2651759462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca0106e80 0x7faca01072a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:36.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.364+0000 7faca7de2700 1 -- 172.21.15.83:0/2651759462 >> 172.21.15.83:0/2651759462 conn(0x7faca0102ac0 msgr2=0x7faca0104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:36.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.365+0000 7faca7de2700 1 -- 172.21.15.83:0/2651759462 shutdown_connections 2024-04-25T11:34:36.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.365+0000 7faca7de2700 1 -- 172.21.15.83:0/2651759462 wait complete. 2024-04-25T11:34:36.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.365+0000 7faca7de2700 1 Processor -- start 2024-04-25T11:34:36.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.365+0000 7faca7de2700 1 -- start start 2024-04-25T11:34:36.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.365+0000 7faca7de2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca0106e80 0x7faca019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:36.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.365+0000 7faca7de2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faca01087a0 con 0x7faca0106e80 2024-04-25T11:34:36.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.365+0000 7faca5b7e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca0106e80 0x7faca019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:36.309 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.365+0000 7faca5b7e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca0106e80 0x7faca019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53358/0 (socket says 172.21.15.83:53358) 2024-04-25T11:34:36.309 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.365+0000 7faca5b7e700 1 -- 172.21.15.83:0/813897838 learned_addr learned my addr 172.21.15.83:0/813897838 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:36.309 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.366+0000 7faca5b7e700 1 -- 172.21.15.83:0/813897838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca019ebf0 con 0x7faca0106e80 2024-04-25T11:34:36.309 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.366+0000 7faca5b7e700 1 --2- 172.21.15.83:0/813897838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca0106e80 0x7faca019e6b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fac9800b7b0 tx=0x7fac9800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:36.309 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.366+0000 7fac9effd700 1 -- 172.21.15.83:0/813897838 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac98020020 con 0x7faca0106e80 2024-04-25T11:34:36.309 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.366+0000 7faca7de2700 1 -- 172.21.15.83:0/813897838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca019ee80 con 0x7faca0106e80 2024-04-25T11:34:36.309 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.366+0000 7fac9effd700 1 -- 172.21.15.83:0/813897838 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac98012430 con 0x7faca0106e80 2024-04-25T11:34:36.309 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.366+0000 7fac9effd700 1 -- 172.21.15.83:0/813897838 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac98017040 con 0x7faca0106e80 2024-04-25T11:34:36.309 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.366+0000 7faca7de2700 1 -- 172.21.15.83:0/813897838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca01a1b70 con 0x7faca0106e80 2024-04-25T11:34:36.309 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.367+0000 7fac9effd700 1 -- 172.21.15.83:0/813897838 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fac98023020 con 0x7faca0106e80 2024-04-25T11:34:36.309 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.367+0000 7faca7de2700 1 -- 172.21.15.83:0/813897838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca00640a0 con 0x7faca0106e80 2024-04-25T11:34:36.309 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.367+0000 7fac9effd700 1 --2- 172.21.15.83:0/813897838 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fac88039560 0x7fac8803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:36.309 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.367+0000 7fac9effd700 1 -- 172.21.15.83:0/813897838 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fac9800f050 con 0x7faca0106e80 2024-04-25T11:34:36.309 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.367+0000 7faca537d700 1 --2- 172.21.15.83:0/813897838 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fac88039560 0x7fac8803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:36.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.368+0000 7faca537d700 1 --2- 172.21.15.83:0/813897838 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fac88039560 0x7fac8803ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fac94006fd0 tx=0x7fac94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:36.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.372+0000 7fac9effd700 1 -- 172.21.15.83:0/813897838 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac9802a020 con 0x7faca0106e80 2024-04-25T11:34:36.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.594+0000 7faca7de2700 1 -- 172.21.15.83:0/813897838 --> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7faca002e2a0 con 0x7fac88039560 2024-04-25T11:34:36.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.601+0000 7fac9effd700 1 -- 172.21.15.83:0/813897838 <== mgr.14120 v2:172.21.15.83:6800/1639787889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7faca002e2a0 con 0x7fac88039560 2024-04-25T11:34:36.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.603+0000 7faca7de2700 1 -- 172.21.15.83:0/813897838 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fac88039560 msgr2=0x7fac8803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:36.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.603+0000 7faca7de2700 1 --2- 172.21.15.83:0/813897838 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fac88039560 0x7fac8803ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fac94006fd0 tx=0x7fac94006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:36.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.603+0000 7faca7de2700 1 -- 172.21.15.83:0/813897838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca0106e80 msgr2=0x7faca019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:36.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.603+0000 7faca7de2700 1 --2- 172.21.15.83:0/813897838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca0106e80 0x7faca019e6b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fac9800b7b0 tx=0x7fac9800be20 comp rx=0 tx=0).stop 2024-04-25T11:34:36.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.603+0000 7faca7de2700 1 -- 172.21.15.83:0/813897838 shutdown_connections 2024-04-25T11:34:36.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.603+0000 7faca7de2700 1 --2- 172.21.15.83:0/813897838 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fac88039560 0x7fac8803ba20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:36.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.603+0000 7faca7de2700 1 --2- 172.21.15.83:0/813897838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca0106e80 0x7faca019e6b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:36.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.603+0000 7faca7de2700 1 -- 172.21.15.83:0/813897838 >> 172.21.15.83:0/813897838 conn(0x7faca0102ac0 msgr2=0x7faca0195390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:36.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.603+0000 7faca7de2700 1 -- 172.21.15.83:0/813897838 shutdown_connections 2024-04-25T11:34:36.311 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:35.603+0000 7faca7de2700 1 -- 172.21.15.83:0/813897838 wait complete. 2024-04-25T11:34:36.311 INFO:teuthology.orchestra.run.smithi083.stdout:Deploying unmanaged mgr service... 2024-04-25T11:34:36.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:36 smithi083 ceph-mon[29330]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:34:36.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:36 smithi083 ceph-mon[29330]: Saving service mon spec with placement count:5 2024-04-25T11:34:36.845 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:36 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' 2024-04-25T11:34:38.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:37 smithi083 ceph-mon[29330]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:34:38.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:37 smithi083 ceph-mon[29330]: Saving service mgr spec with placement count:2 2024-04-25T11:34:38.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:37 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' 2024-04-25T11:34:38.296 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:37 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' 2024-04-25T11:34:38.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T11:34:38.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.746+0000 7f9b5a3b1700 1 Processor -- start 2024-04-25T11:34:38.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.746+0000 7f9b5a3b1700 1 -- start start 2024-04-25T11:34:38.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.746+0000 7f9b5a3b1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b54108600 0x7f9b54108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:38.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.746+0000 7f9b5a3b1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9b54108ff0 con 0x7f9b54108600 2024-04-25T11:34:38.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.746+0000 7f9b53fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b54108600 0x7f9b54108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:38.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.746+0000 7f9b53fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b54108600 0x7f9b54108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53360/0 (socket says 172.21.15.83:53360) 2024-04-25T11:34:38.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.746+0000 7f9b53fff700 1 -- 172.21.15.83:0/1782339684 learned_addr learned my addr 172.21.15.83:0/1782339684 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:38.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.746+0000 7f9b53fff700 1 -- 172.21.15.83:0/1782339684 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b541097b0 con 0x7f9b54108600 2024-04-25T11:34:38.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.746+0000 7f9b53fff700 1 --2- 172.21.15.83:0/1782339684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b54108600 0x7f9b54108a20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9b480098a0 tx=0x7f9b48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1322b16648fbaf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:38.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.746+0000 7f9b52ffd700 1 -- 172.21.15.83:0/1782339684 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b48004160 con 0x7f9b54108600 2024-04-25T11:34:38.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.746+0000 7f9b52ffd700 1 -- 172.21.15.83:0/1782339684 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9b48004300 con 0x7f9b54108600 2024-04-25T11:34:38.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.746+0000 7f9b52ffd700 1 -- 172.21.15.83:0/1782339684 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b48011af0 con 0x7f9b54108600 2024-04-25T11:34:38.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.747+0000 7f9b5a3b1700 1 -- 172.21.15.83:0/1782339684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b54108600 msgr2=0x7f9b54108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:38.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.747+0000 7f9b5a3b1700 1 --2- 172.21.15.83:0/1782339684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b54108600 0x7f9b54108a20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9b480098a0 tx=0x7f9b48009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:38.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.747+0000 7f9b5a3b1700 1 -- 172.21.15.83:0/1782339684 shutdown_connections 2024-04-25T11:34:38.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.747+0000 7f9b5a3b1700 1 --2- 172.21.15.83:0/1782339684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b54108600 0x7f9b54108a20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:38.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.747+0000 7f9b5a3b1700 1 -- 172.21.15.83:0/1782339684 >> 172.21.15.83:0/1782339684 conn(0x7f9b54103bc0 msgr2=0x7f9b54105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:38.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.747+0000 7f9b5a3b1700 1 -- 172.21.15.83:0/1782339684 shutdown_connections 2024-04-25T11:34:38.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.747+0000 7f9b5a3b1700 1 -- 172.21.15.83:0/1782339684 wait complete. 2024-04-25T11:34:38.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.748+0000 7f9b5a3b1700 1 Processor -- start 2024-04-25T11:34:38.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.748+0000 7f9b5a3b1700 1 -- start start 2024-04-25T11:34:38.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.748+0000 7f9b5a3b1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b54108600 0x7f9b5407db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:38.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.748+0000 7f9b5a3b1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5410c9e0 con 0x7f9b54108600 2024-04-25T11:34:38.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.748+0000 7f9b53fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b54108600 0x7f9b5407db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:38.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.748+0000 7f9b53fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b54108600 0x7f9b5407db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53376/0 (socket says 172.21.15.83:53376) 2024-04-25T11:34:38.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.748+0000 7f9b53fff700 1 -- 172.21.15.83:0/1837674518 learned_addr learned my addr 172.21.15.83:0/1837674518 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:38.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.748+0000 7f9b53fff700 1 -- 172.21.15.83:0/1837674518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5407e090 con 0x7f9b54108600 2024-04-25T11:34:38.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.748+0000 7f9b53fff700 1 --2- 172.21.15.83:0/1837674518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b54108600 0x7f9b5407db50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b48000c00 tx=0x7f9b48011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:38.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.748+0000 7f9b517fa700 1 -- 172.21.15.83:0/1837674518 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b48003b20 con 0x7f9b54108600 2024-04-25T11:34:38.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.749+0000 7f9b517fa700 1 -- 172.21.15.83:0/1837674518 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9b4801a8c0 con 0x7f9b54108600 2024-04-25T11:34:38.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.749+0000 7f9b517fa700 1 -- 172.21.15.83:0/1837674518 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b48024940 con 0x7f9b54108600 2024-04-25T11:34:38.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.749+0000 7f9b5a3b1700 1 -- 172.21.15.83:0/1837674518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5407c280 con 0x7f9b54108600 2024-04-25T11:34:38.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.749+0000 7f9b5a3b1700 1 -- 172.21.15.83:0/1837674518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5407c7c0 con 0x7f9b54108600 2024-04-25T11:34:38.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.749+0000 7f9b517fa700 1 -- 172.21.15.83:0/1837674518 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9b48022020 con 0x7f9b54108600 2024-04-25T11:34:38.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.750+0000 7f9b5a3b1700 1 -- 172.21.15.83:0/1837674518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b540640a0 con 0x7f9b54108600 2024-04-25T11:34:38.306 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.750+0000 7f9b517fa700 1 --2- 172.21.15.83:0/1837674518 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f9b3c0395b0 0x7f9b3c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:38.306 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.750+0000 7f9b517fa700 1 -- 172.21.15.83:0/1837674518 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9b4804d830 con 0x7f9b54108600 2024-04-25T11:34:38.306 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.750+0000 7f9b537fe700 1 --2- 172.21.15.83:0/1837674518 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f9b3c0395b0 0x7f9b3c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:38.306 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.750+0000 7f9b537fe700 1 --2- 172.21.15.83:0/1837674518 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f9b3c0395b0 0x7f9b3c03ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9b44006fd0 tx=0x7f9b44006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:38.306 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.755+0000 7f9b517fa700 1 -- 172.21.15.83:0/1837674518 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b4800be50 con 0x7f9b54108600 2024-04-25T11:34:38.306 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.973+0000 7f9b5a3b1700 1 -- 172.21.15.83:0/1837674518 --> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f9b54104da0 con 0x7f9b3c0395b0 2024-04-25T11:34:38.306 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.980+0000 7f9b517fa700 1 -- 172.21.15.83:0/1837674518 <== mgr.14120 v2:172.21.15.83:6800/1639787889 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9b54104da0 con 0x7f9b3c0395b0 2024-04-25T11:34:38.306 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.982+0000 7f9b5a3b1700 1 -- 172.21.15.83:0/1837674518 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f9b3c0395b0 msgr2=0x7f9b3c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:38.306 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.982+0000 7f9b5a3b1700 1 --2- 172.21.15.83:0/1837674518 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f9b3c0395b0 0x7f9b3c03ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9b44006fd0 tx=0x7f9b44006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:38.306 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.982+0000 7f9b5a3b1700 1 -- 172.21.15.83:0/1837674518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b54108600 msgr2=0x7f9b5407db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:38.306 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.982+0000 7f9b5a3b1700 1 --2- 172.21.15.83:0/1837674518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b54108600 0x7f9b5407db50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9b48000c00 tx=0x7f9b48011b70 comp rx=0 tx=0).stop 2024-04-25T11:34:38.306 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.982+0000 7f9b5a3b1700 1 -- 172.21.15.83:0/1837674518 shutdown_connections 2024-04-25T11:34:38.306 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.982+0000 7f9b5a3b1700 1 --2- 172.21.15.83:0/1837674518 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f9b3c0395b0 0x7f9b3c03ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:38.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.982+0000 7f9b5a3b1700 1 --2- 172.21.15.83:0/1837674518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b54108600 0x7f9b5407db50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:38.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.982+0000 7f9b5a3b1700 1 -- 172.21.15.83:0/1837674518 >> 172.21.15.83:0/1837674518 conn(0x7f9b54103bc0 msgr2=0x7f9b54104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:38.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.982+0000 7f9b5a3b1700 1 -- 172.21.15.83:0/1837674518 shutdown_connections 2024-04-25T11:34:38.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:36.982+0000 7f9b5a3b1700 1 -- 172.21.15.83:0/1837674518 wait complete. 2024-04-25T11:34:39.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.765+0000 7f9a48c9e700 1 Processor -- start 2024-04-25T11:34:39.526 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.765+0000 7f9a48c9e700 1 -- start start 2024-04-25T11:34:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.765+0000 7f9a48c9e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441085b0 0x7f9a441089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.765+0000 7f9a48c9e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a44108fa0 con 0x7f9a441085b0 2024-04-25T11:34:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.766+0000 7f9a4259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441085b0 0x7f9a441089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.766+0000 7f9a4259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441085b0 0x7f9a441089d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53378/0 (socket says 172.21.15.83:53378) 2024-04-25T11:34:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.766+0000 7f9a4259c700 1 -- 172.21.15.83:0/902040679 learned_addr learned my addr 172.21.15.83:0/902040679 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.766+0000 7f9a4259c700 1 -- 172.21.15.83:0/902040679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a44109760 con 0x7f9a441085b0 2024-04-25T11:34:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.766+0000 7f9a4259c700 1 --2- 172.21.15.83:0/902040679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441085b0 0x7f9a441089d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9a380098a0 tx=0x7f9a38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae681c67b0a3ec81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.766+0000 7f9a4159a700 1 -- 172.21.15.83:0/902040679 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a38004160 con 0x7f9a441085b0 2024-04-25T11:34:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.766+0000 7f9a4159a700 1 -- 172.21.15.83:0/902040679 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a38004300 con 0x7f9a441085b0 2024-04-25T11:34:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.766+0000 7f9a4159a700 1 -- 172.21.15.83:0/902040679 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a38011a20 con 0x7f9a441085b0 2024-04-25T11:34:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.767+0000 7f9a48c9e700 1 -- 172.21.15.83:0/902040679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441085b0 msgr2=0x7f9a441089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.767+0000 7f9a48c9e700 1 --2- 172.21.15.83:0/902040679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441085b0 0x7f9a441089d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9a380098a0 tx=0x7f9a38009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.767+0000 7f9a48c9e700 1 -- 172.21.15.83:0/902040679 shutdown_connections 2024-04-25T11:34:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.767+0000 7f9a48c9e700 1 --2- 172.21.15.83:0/902040679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441085b0 0x7f9a441089d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.767+0000 7f9a48c9e700 1 -- 172.21.15.83:0/902040679 >> 172.21.15.83:0/902040679 conn(0x7f9a44103b70 msgr2=0x7f9a44105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.767+0000 7f9a48c9e700 1 -- 172.21.15.83:0/902040679 shutdown_connections 2024-04-25T11:34:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.767+0000 7f9a48c9e700 1 -- 172.21.15.83:0/902040679 wait complete. 2024-04-25T11:34:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.768+0000 7f9a48c9e700 1 Processor -- start 2024-04-25T11:34:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.768+0000 7f9a48c9e700 1 -- start start 2024-04-25T11:34:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.768+0000 7f9a48c9e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441085b0 0x7f9a4419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.768+0000 7f9a48c9e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4410cb10 con 0x7f9a441085b0 2024-04-25T11:34:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.768+0000 7f9a4259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441085b0 0x7f9a4419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.768+0000 7f9a4259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441085b0 0x7f9a4419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53382/0 (socket says 172.21.15.83:53382) 2024-04-25T11:34:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.768+0000 7f9a4259c700 1 -- 172.21.15.83:0/1413588084 learned_addr learned my addr 172.21.15.83:0/1413588084 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.769+0000 7f9a4259c700 1 -- 172.21.15.83:0/1413588084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4419c970 con 0x7f9a441085b0 2024-04-25T11:34:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.769+0000 7f9a4259c700 1 --2- 172.21.15.83:0/1413588084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441085b0 0x7f9a4419c430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009520 tx=0x7f9a38011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.769+0000 7f9a337fe700 1 -- 172.21.15.83:0/1413588084 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a380039e0 con 0x7f9a441085b0 2024-04-25T11:34:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.769+0000 7f9a48c9e700 1 -- 172.21.15.83:0/1413588084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4419cc00 con 0x7f9a441085b0 2024-04-25T11:34:39.528 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.769+0000 7f9a337fe700 1 -- 172.21.15.83:0/1413588084 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a3801b8f0 con 0x7f9a441085b0 2024-04-25T11:34:39.529 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.769+0000 7f9a337fe700 1 -- 172.21.15.83:0/1413588084 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a38019770 con 0x7f9a441085b0 2024-04-25T11:34:39.529 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.769+0000 7f9a48c9e700 1 -- 172.21.15.83:0/1413588084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4419f8f0 con 0x7f9a441085b0 2024-04-25T11:34:39.529 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.770+0000 7f9a337fe700 1 -- 172.21.15.83:0/1413588084 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9a38019a70 con 0x7f9a441085b0 2024-04-25T11:34:39.529 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.770+0000 7f9a337fe700 1 --2- 172.21.15.83:0/1413588084 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f9a2c0395b0 0x7f9a2c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:39.529 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.770+0000 7f9a337fe700 1 -- 172.21.15.83:0/1413588084 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9a3804e070 con 0x7f9a441085b0 2024-04-25T11:34:39.529 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.770+0000 7f9a48c9e700 1 -- 172.21.15.83:0/1413588084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4410cc90 con 0x7f9a441085b0 2024-04-25T11:34:39.529 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.770+0000 7f9a41d9b700 1 --2- 172.21.15.83:0/1413588084 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f9a2c0395b0 0x7f9a2c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:39.529 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.771+0000 7f9a41d9b700 1 --2- 172.21.15.83:0/1413588084 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f9a2c0395b0 0x7f9a2c03ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9a34006fd0 tx=0x7f9a34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:39.529 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.775+0000 7f9a337fe700 1 -- 172.21.15.83:0/1413588084 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a3801f030 con 0x7f9a441085b0 2024-04-25T11:34:39.529 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.980+0000 7f9a48c9e700 1 -- 172.21.15.83:0/1413588084 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f9a4419fd30 con 0x7f9a441085b0 2024-04-25T11:34:39.529 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.982+0000 7f9a337fe700 1 -- 172.21.15.83:0/1413588084 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f9a38019db0 con 0x7f9a441085b0 2024-04-25T11:34:39.529 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.985+0000 7f9a48c9e700 1 -- 172.21.15.83:0/1413588084 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f9a2c0395b0 msgr2=0x7f9a2c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:39.529 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.985+0000 7f9a48c9e700 1 --2- 172.21.15.83:0/1413588084 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f9a2c0395b0 0x7f9a2c03ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9a34006fd0 tx=0x7f9a34006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:39.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.985+0000 7f9a48c9e700 1 -- 172.21.15.83:0/1413588084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441085b0 msgr2=0x7f9a4419c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:39.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.985+0000 7f9a48c9e700 1 --2- 172.21.15.83:0/1413588084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441085b0 0x7f9a4419c430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009520 tx=0x7f9a38011aa0 comp rx=0 tx=0).stop 2024-04-25T11:34:39.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.985+0000 7f9a48c9e700 1 -- 172.21.15.83:0/1413588084 shutdown_connections 2024-04-25T11:34:39.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.985+0000 7f9a48c9e700 1 --2- 172.21.15.83:0/1413588084 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f9a2c0395b0 0x7f9a2c03ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:39.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.986+0000 7f9a48c9e700 1 --2- 172.21.15.83:0/1413588084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a441085b0 0x7f9a4419c430 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:39.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.986+0000 7f9a48c9e700 1 -- 172.21.15.83:0/1413588084 >> 172.21.15.83:0/1413588084 conn(0x7f9a44103b70 msgr2=0x7f9a441046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:39.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.986+0000 7f9a48c9e700 1 -- 172.21.15.83:0/1413588084 shutdown_connections 2024-04-25T11:34:39.530 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:38.986+0000 7f9a48c9e700 1 -- 172.21.15.83:0/1413588084 wait complete. 2024-04-25T11:34:39.855 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:39 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' 2024-04-25T11:34:39.855 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:39 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1413588084' entity='client.admin' 2024-04-25T11:34:41.245 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:41 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/4205219552' entity='client.admin' 2024-04-25T11:34:41.245 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:41 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' 2024-04-25T11:34:41.252 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.967+0000 7efd37f7c700 1 Processor -- start 2024-04-25T11:34:41.252 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.967+0000 7efd37f7c700 1 -- start start 2024-04-25T11:34:41.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.967+0000 7efd37f7c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd301074b0 0x7efd301078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:41.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.967+0000 7efd37f7c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd30107ea0 con 0x7efd301074b0 2024-04-25T11:34:41.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.968+0000 7efd35d18700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd301074b0 0x7efd301078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:41.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.968+0000 7efd35d18700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd301074b0 0x7efd301078d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53396/0 (socket says 172.21.15.83:53396) 2024-04-25T11:34:41.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.968+0000 7efd35d18700 1 -- 172.21.15.83:0/915730413 learned_addr learned my addr 172.21.15.83:0/915730413 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:41.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.968+0000 7efd35d18700 1 -- 172.21.15.83:0/915730413 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd301086b0 con 0x7efd301074b0 2024-04-25T11:34:41.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.968+0000 7efd35d18700 1 --2- 172.21.15.83:0/915730413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd301074b0 0x7efd301078d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efd2c0098a0 tx=0x7efd2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c187dfee78fb03f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:41.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.968+0000 7efd34d16700 1 -- 172.21.15.83:0/915730413 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd2c004160 con 0x7efd301074b0 2024-04-25T11:34:41.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.968+0000 7efd34d16700 1 -- 172.21.15.83:0/915730413 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd2c004300 con 0x7efd301074b0 2024-04-25T11:34:41.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.968+0000 7efd34d16700 1 -- 172.21.15.83:0/915730413 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd2c011a20 con 0x7efd301074b0 2024-04-25T11:34:41.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.969+0000 7efd37f7c700 1 -- 172.21.15.83:0/915730413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd301074b0 msgr2=0x7efd301078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:41.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.969+0000 7efd37f7c700 1 --2- 172.21.15.83:0/915730413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd301074b0 0x7efd301078d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efd2c0098a0 tx=0x7efd2c009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.969+0000 7efd37f7c700 1 -- 172.21.15.83:0/915730413 shutdown_connections 2024-04-25T11:34:41.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.969+0000 7efd37f7c700 1 --2- 172.21.15.83:0/915730413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd301074b0 0x7efd301078d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.969+0000 7efd37f7c700 1 -- 172.21.15.83:0/915730413 >> 172.21.15.83:0/915730413 conn(0x7efd30102a30 msgr2=0x7efd30104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:41.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.969+0000 7efd37f7c700 1 -- 172.21.15.83:0/915730413 shutdown_connections 2024-04-25T11:34:41.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.969+0000 7efd37f7c700 1 -- 172.21.15.83:0/915730413 wait complete. 2024-04-25T11:34:41.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.969+0000 7efd37f7c700 1 Processor -- start 2024-04-25T11:34:41.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.969+0000 7efd37f7c700 1 -- start start 2024-04-25T11:34:41.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.970+0000 7efd37f7c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd301074b0 0x7efd3019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:41.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.970+0000 7efd37f7c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd3010ba60 con 0x7efd301074b0 2024-04-25T11:34:41.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.970+0000 7efd35d18700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd301074b0 0x7efd3019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:41.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.970+0000 7efd35d18700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd301074b0 0x7efd3019b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53406/0 (socket says 172.21.15.83:53406) 2024-04-25T11:34:41.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.970+0000 7efd35d18700 1 -- 172.21.15.83:0/4205219552 learned_addr learned my addr 172.21.15.83:0/4205219552 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:41.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.970+0000 7efd35d18700 1 -- 172.21.15.83:0/4205219552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3019b960 con 0x7efd301074b0 2024-04-25T11:34:41.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.970+0000 7efd35d18700 1 --2- 172.21.15.83:0/4205219552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd301074b0 0x7efd3019b420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efd2c000c00 tx=0x7efd2c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:41.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.970+0000 7efd26ffd700 1 -- 172.21.15.83:0/4205219552 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd2c003aa0 con 0x7efd301074b0 2024-04-25T11:34:41.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.970+0000 7efd26ffd700 1 -- 172.21.15.83:0/4205219552 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd2c01b8f0 con 0x7efd301074b0 2024-04-25T11:34:41.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.970+0000 7efd37f7c700 1 -- 172.21.15.83:0/4205219552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3019bbf0 con 0x7efd301074b0 2024-04-25T11:34:41.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.970+0000 7efd26ffd700 1 -- 172.21.15.83:0/4205219552 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd2c023890 con 0x7efd301074b0 2024-04-25T11:34:41.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.970+0000 7efd37f7c700 1 -- 172.21.15.83:0/4205219552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3019e8e0 con 0x7efd301074b0 2024-04-25T11:34:41.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.971+0000 7efd26ffd700 1 -- 172.21.15.83:0/4205219552 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7efd2c01b440 con 0x7efd301074b0 2024-04-25T11:34:41.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.971+0000 7efd37f7c700 1 -- 172.21.15.83:0/4205219552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd3010bbe0 con 0x7efd301074b0 2024-04-25T11:34:41.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.971+0000 7efd26ffd700 1 --2- 172.21.15.83:0/4205219552 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7efd1c0395b0 0x7efd1c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:41.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.971+0000 7efd26ffd700 1 -- 172.21.15.83:0/4205219552 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efd2c04de90 con 0x7efd301074b0 2024-04-25T11:34:41.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.971+0000 7efd35517700 1 --2- 172.21.15.83:0/4205219552 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7efd1c0395b0 0x7efd1c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:41.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.972+0000 7efd35517700 1 --2- 172.21.15.83:0/4205219552 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7efd1c0395b0 0x7efd1c03ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efd20006fd0 tx=0x7efd20006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:41.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:39.976+0000 7efd26ffd700 1 -- 172.21.15.83:0/4205219552 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd2c01f030 con 0x7efd301074b0 2024-04-25T11:34:41.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:40.183+0000 7efd37f7c700 1 -- 172.21.15.83:0/4205219552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7efd3019eea0 con 0x7efd301074b0 2024-04-25T11:34:41.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:40.185+0000 7efd26ffd700 1 -- 172.21.15.83:0/4205219552 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7efd2c00be50 con 0x7efd301074b0 2024-04-25T11:34:41.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:40.187+0000 7efd37f7c700 1 -- 172.21.15.83:0/4205219552 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7efd1c0395b0 msgr2=0x7efd1c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:41.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:40.187+0000 7efd37f7c700 1 --2- 172.21.15.83:0/4205219552 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7efd1c0395b0 0x7efd1c03ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efd20006fd0 tx=0x7efd20006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:41.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:40.188+0000 7efd37f7c700 1 -- 172.21.15.83:0/4205219552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd301074b0 msgr2=0x7efd3019b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:41.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:40.188+0000 7efd37f7c700 1 --2- 172.21.15.83:0/4205219552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd301074b0 0x7efd3019b420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efd2c000c00 tx=0x7efd2c011aa0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:40.188+0000 7efd37f7c700 1 -- 172.21.15.83:0/4205219552 shutdown_connections 2024-04-25T11:34:41.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:40.188+0000 7efd37f7c700 1 --2- 172.21.15.83:0/4205219552 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7efd1c0395b0 0x7efd1c03ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:40.188+0000 7efd37f7c700 1 --2- 172.21.15.83:0/4205219552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd301074b0 0x7efd3019b420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:41.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:40.188+0000 7efd37f7c700 1 -- 172.21.15.83:0/4205219552 >> 172.21.15.83:0/4205219552 conn(0x7efd30102a30 msgr2=0x7efd30103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:41.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:40.188+0000 7efd37f7c700 1 -- 172.21.15.83:0/4205219552 shutdown_connections 2024-04-25T11:34:41.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:40.188+0000 7efd37f7c700 1 -- 172.21.15.83:0/4205219552 wait complete. 2024-04-25T11:34:41.256 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling the dashboard module... 2024-04-25T11:34:43.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:42 smithi083 ceph-mon[29330]: from='mgr.14120 172.21.15.83:0/844117575' entity='mgr.x' 2024-04-25T11:34:43.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:42 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2206966059' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T11:34:43.124 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:43 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: ignoring --setuser ceph since I am not root 2024-04-25T11:34:43.125 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:43 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: ignoring --setgroup ceph since I am not root 2024-04-25T11:34:43.125 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:43 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:43.135+0000 7f96c015d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:34:43.407 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:43 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:43.322+0000 7f96c015d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:34:43.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.695+0000 7fad8fd28700 1 Processor -- start 2024-04-25T11:34:43.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.695+0000 7fad8fd28700 1 -- start start 2024-04-25T11:34:43.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.696+0000 7fad8fd28700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad88109060 0x7fad8810b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:43.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.696+0000 7fad8fd28700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad8807be50 con 0x7fad88109060 2024-04-25T11:34:43.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.696+0000 7fad8dac4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad88109060 0x7fad8810b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:43.531 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.696+0000 7fad8dac4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad88109060 0x7fad8810b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53414/0 (socket says 172.21.15.83:53414) 2024-04-25T11:34:43.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.696+0000 7fad8dac4700 1 -- 172.21.15.83:0/2557530435 learned_addr learned my addr 172.21.15.83:0/2557530435 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:43.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.696+0000 7fad8dac4700 1 -- 172.21.15.83:0/2557530435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8810b990 con 0x7fad88109060 2024-04-25T11:34:43.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.696+0000 7fad8dac4700 1 --2- 172.21.15.83:0/2557530435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad88109060 0x7fad8810b450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fad840098a0 tx=0x7fad84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103e96ab393160be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:43.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.697+0000 7fad8cac2700 1 -- 172.21.15.83:0/2557530435 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad84004160 con 0x7fad88109060 2024-04-25T11:34:43.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.697+0000 7fad8cac2700 1 -- 172.21.15.83:0/2557530435 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad84004300 con 0x7fad88109060 2024-04-25T11:34:43.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.697+0000 7fad8cac2700 1 -- 172.21.15.83:0/2557530435 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad84011a20 con 0x7fad88109060 2024-04-25T11:34:43.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.697+0000 7fad8fd28700 1 -- 172.21.15.83:0/2557530435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad88109060 msgr2=0x7fad8810b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:43.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.697+0000 7fad8fd28700 1 --2- 172.21.15.83:0/2557530435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad88109060 0x7fad8810b450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fad840098a0 tx=0x7fad84009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.697+0000 7fad8fd28700 1 -- 172.21.15.83:0/2557530435 shutdown_connections 2024-04-25T11:34:43.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.697+0000 7fad8fd28700 1 --2- 172.21.15.83:0/2557530435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad88109060 0x7fad8810b450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.697+0000 7fad8fd28700 1 -- 172.21.15.83:0/2557530435 >> 172.21.15.83:0/2557530435 conn(0x7fad88102a70 msgr2=0x7fad88104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:43.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.697+0000 7fad8fd28700 1 -- 172.21.15.83:0/2557530435 shutdown_connections 2024-04-25T11:34:43.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.697+0000 7fad8fd28700 1 -- 172.21.15.83:0/2557530435 wait complete. 2024-04-25T11:34:43.532 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.698+0000 7fad8fd28700 1 Processor -- start 2024-04-25T11:34:43.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.698+0000 7fad8fd28700 1 -- start start 2024-04-25T11:34:43.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.698+0000 7fad8fd28700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad88109060 0x7fad881088e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:43.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.698+0000 7fad8fd28700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad8810c990 con 0x7fad88109060 2024-04-25T11:34:43.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.698+0000 7fad8dac4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad88109060 0x7fad881088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:43.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.698+0000 7fad8dac4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad88109060 0x7fad881088e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53422/0 (socket says 172.21.15.83:53422) 2024-04-25T11:34:43.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.698+0000 7fad8dac4700 1 -- 172.21.15.83:0/2206966059 learned_addr learned my addr 172.21.15.83:0/2206966059 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:43.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.699+0000 7fad8dac4700 1 -- 172.21.15.83:0/2206966059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad88108e20 con 0x7fad88109060 2024-04-25T11:34:43.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.699+0000 7fad8dac4700 1 --2- 172.21.15.83:0/2206966059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad88109060 0x7fad881088e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fad840116e0 tx=0x7fad84011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:43.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.699+0000 7fad7effd700 1 -- 172.21.15.83:0/2206966059 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad84003b20 con 0x7fad88109060 2024-04-25T11:34:43.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.699+0000 7fad7effd700 1 -- 172.21.15.83:0/2206966059 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8401b8f0 con 0x7fad88109060 2024-04-25T11:34:43.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.699+0000 7fad7effd700 1 -- 172.21.15.83:0/2206966059 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad840238c0 con 0x7fad88109060 2024-04-25T11:34:43.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.699+0000 7fad8fd28700 1 -- 172.21.15.83:0/2206966059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad88107010 con 0x7fad88109060 2024-04-25T11:34:43.533 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.699+0000 7fad8fd28700 1 -- 172.21.15.83:0/2206966059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad88107550 con 0x7fad88109060 2024-04-25T11:34:43.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.700+0000 7fad7effd700 1 -- 172.21.15.83:0/2206966059 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fad84023a80 con 0x7fad88109060 2024-04-25T11:34:43.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.700+0000 7fad8fd28700 1 -- 172.21.15.83:0/2206966059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad880640a0 con 0x7fad88109060 2024-04-25T11:34:43.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.700+0000 7fad7effd700 1 --2- 172.21.15.83:0/2206966059 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fad740395b0 0x7fad7403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:43.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.700+0000 7fad7effd700 1 -- 172.21.15.83:0/2206966059 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fad8404dfc0 con 0x7fad88109060 2024-04-25T11:34:43.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.700+0000 7fad8d2c3700 1 --2- 172.21.15.83:0/2206966059 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fad740395b0 0x7fad7403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:43.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.701+0000 7fad8d2c3700 1 --2- 172.21.15.83:0/2206966059 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fad740395b0 0x7fad7403ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fad78006fd0 tx=0x7fad78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:43.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.705+0000 7fad7effd700 1 -- 172.21.15.83:0/2206966059 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad8401f030 con 0x7fad88109060 2024-04-25T11:34:43.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:41.973+0000 7fad8fd28700 1 -- 172.21.15.83:0/2206966059 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fad881077d0 con 0x7fad88109060 2024-04-25T11:34:43.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:42.958+0000 7fad7effd700 1 -- 172.21.15.83:0/2206966059 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fad8401a3d0 con 0x7fad88109060 2024-04-25T11:34:43.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:42.959+0000 7fad7effd700 1 -- 172.21.15.83:0/2206966059 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fad84022bc0 con 0x7fad88109060 2024-04-25T11:34:43.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:42.960+0000 7fad8fd28700 1 -- 172.21.15.83:0/2206966059 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fad740395b0 msgr2=0x7fad7403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:43.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:42.960+0000 7fad8fd28700 1 --2- 172.21.15.83:0/2206966059 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fad740395b0 0x7fad7403ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fad78006fd0 tx=0x7fad78006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:43.534 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:42.960+0000 7fad8fd28700 1 -- 172.21.15.83:0/2206966059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad88109060 msgr2=0x7fad881088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:43.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:42.960+0000 7fad8fd28700 1 --2- 172.21.15.83:0/2206966059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad88109060 0x7fad881088e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fad840116e0 tx=0x7fad84011aa0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:42.960+0000 7fad8fd28700 1 -- 172.21.15.83:0/2206966059 shutdown_connections 2024-04-25T11:34:43.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:42.960+0000 7fad8fd28700 1 --2- 172.21.15.83:0/2206966059 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fad740395b0 0x7fad7403ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:42.960+0000 7fad8fd28700 1 --2- 172.21.15.83:0/2206966059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad88109060 0x7fad881088e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:43.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:42.960+0000 7fad8fd28700 1 -- 172.21.15.83:0/2206966059 >> 172.21.15.83:0/2206966059 conn(0x7fad88102a70 msgr2=0x7fad88103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:43.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:42.960+0000 7fad8fd28700 1 -- 172.21.15.83:0/2206966059 shutdown_connections 2024-04-25T11:34:43.535 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:42.961+0000 7fad8fd28700 1 -- 172.21.15.83:0/2206966059 wait complete. 2024-04-25T11:34:43.669 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:43 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:43.423+0000 7f96c015d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:34:43.951 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:43 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:43.737+0000 7f96c015d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:34:43.951 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:43 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:43.822+0000 7f96c015d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:34:44.286 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:43 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2206966059' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T11:34:44.286 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:43 smithi083 ceph-mon[29330]: mgrmap e9: x(active, since 19s) 2024-04-25T11:34:45.206 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:44 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2555603351' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T11:34:45.207 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:45 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:45.017+0000 7f96c015d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:34:45.207 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:45 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:45.188+0000 7f96c015d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:34:45.215 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:34:45.215 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T11:34:45.215 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T11:34:45.215 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T11:34:45.216 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T11:34:45.216 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:34:45.216 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.398+0000 7f2e1a7b8700 1 Processor -- start 2024-04-25T11:34:45.216 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.398+0000 7f2e1a7b8700 1 -- start start 2024-04-25T11:34:45.216 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.398+0000 7f2e1a7b8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14109100 0x7f2e1410b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:45.216 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.398+0000 7f2e1a7b8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e14063ca0 con 0x7f2e14109100 2024-04-25T11:34:45.216 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.398+0000 7f2e13fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14109100 0x7f2e1410b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:45.216 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.398+0000 7f2e13fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14109100 0x7f2e1410b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35792/0 (socket says 172.21.15.83:35792) 2024-04-25T11:34:45.216 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.398+0000 7f2e13fff700 1 -- 172.21.15.83:0/4226796783 learned_addr learned my addr 172.21.15.83:0/4226796783 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:45.216 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.398+0000 7f2e13fff700 1 -- 172.21.15.83:0/4226796783 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1410ba30 con 0x7f2e14109100 2024-04-25T11:34:45.216 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.399+0000 7f2e13fff700 1 --2- 172.21.15.83:0/4226796783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14109100 0x7f2e1410b4f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e080098a0 tx=0x7f2e08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9150f91a710af2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:45.216 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.399+0000 7f2e12ffd700 1 -- 172.21.15.83:0/4226796783 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e08004160 con 0x7f2e14109100 2024-04-25T11:34:45.216 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.399+0000 7f2e12ffd700 1 -- 172.21.15.83:0/4226796783 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e08004300 con 0x7f2e14109100 2024-04-25T11:34:45.217 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.399+0000 7f2e12ffd700 1 -- 172.21.15.83:0/4226796783 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e08011a20 con 0x7f2e14109100 2024-04-25T11:34:45.217 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.399+0000 7f2e1a7b8700 1 -- 172.21.15.83:0/4226796783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14109100 msgr2=0x7f2e1410b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:45.217 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.399+0000 7f2e1a7b8700 1 --2- 172.21.15.83:0/4226796783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14109100 0x7f2e1410b4f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e080098a0 tx=0x7f2e08009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:45.217 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.399+0000 7f2e1a7b8700 1 -- 172.21.15.83:0/4226796783 shutdown_connections 2024-04-25T11:34:45.217 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.399+0000 7f2e1a7b8700 1 --2- 172.21.15.83:0/4226796783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14109100 0x7f2e1410b4f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:45.217 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.399+0000 7f2e1a7b8700 1 -- 172.21.15.83:0/4226796783 >> 172.21.15.83:0/4226796783 conn(0x7f2e14102ab0 msgr2=0x7f2e14104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:45.217 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.399+0000 7f2e1a7b8700 1 -- 172.21.15.83:0/4226796783 shutdown_connections 2024-04-25T11:34:45.217 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.399+0000 7f2e1a7b8700 1 -- 172.21.15.83:0/4226796783 wait complete. 2024-04-25T11:34:45.217 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.400+0000 7f2e1a7b8700 1 Processor -- start 2024-04-25T11:34:45.217 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.400+0000 7f2e1a7b8700 1 -- start start 2024-04-25T11:34:45.217 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.400+0000 7f2e1a7b8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14109100 0x7f2e1419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:45.217 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.400+0000 7f2e1a7b8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1410c980 con 0x7f2e14109100 2024-04-25T11:34:45.217 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.400+0000 7f2e13fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14109100 0x7f2e1419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:45.217 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.400+0000 7f2e13fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14109100 0x7f2e1419c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35806/0 (socket says 172.21.15.83:35806) 2024-04-25T11:34:45.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.400+0000 7f2e13fff700 1 -- 172.21.15.83:0/2555603351 learned_addr learned my addr 172.21.15.83:0/2555603351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:45.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.401+0000 7f2e13fff700 1 -- 172.21.15.83:0/2555603351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1419c950 con 0x7f2e14109100 2024-04-25T11:34:45.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.401+0000 7f2e13fff700 1 --2- 172.21.15.83:0/2555603351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14109100 0x7f2e1419c410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2e080116e0 tx=0x7f2e08011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:45.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.401+0000 7f2e117fa700 1 -- 172.21.15.83:0/2555603351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e08003a60 con 0x7f2e14109100 2024-04-25T11:34:45.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.401+0000 7f2e117fa700 1 -- 172.21.15.83:0/2555603351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e0801b8f0 con 0x7f2e14109100 2024-04-25T11:34:45.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.401+0000 7f2e1a7b8700 1 -- 172.21.15.83:0/2555603351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1419cbe0 con 0x7f2e14109100 2024-04-25T11:34:45.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.401+0000 7f2e117fa700 1 -- 172.21.15.83:0/2555603351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e08019860 con 0x7f2e14109100 2024-04-25T11:34:45.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.401+0000 7f2e1a7b8700 1 -- 172.21.15.83:0/2555603351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e1419f8d0 con 0x7f2e14109100 2024-04-25T11:34:45.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.402+0000 7f2e117fa700 1 -- 172.21.15.83:0/2555603351 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f2e08019b60 con 0x7f2e14109100 2024-04-25T11:34:45.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.402+0000 7f2e117fa700 1 --2- 172.21.15.83:0/2555603351 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f2dfc0395b0 0x7f2dfc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:45.218 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.402+0000 7f2e117fa700 1 -- 172.21.15.83:0/2555603351 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2e0804ed30 con 0x7f2e14109100 2024-04-25T11:34:45.219 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.402+0000 7f2e137fe700 1 -- 172.21.15.83:0/2555603351 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f2dfc0395b0 msgr2=0x7f2dfc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1639787889 2024-04-25T11:34:45.219 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.402+0000 7f2e137fe700 1 --2- 172.21.15.83:0/2555603351 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f2dfc0395b0 0x7f2dfc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:34:45.219 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.402+0000 7f2e1a7b8700 1 -- 172.21.15.83:0/2555603351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e140640a0 con 0x7f2e14109100 2024-04-25T11:34:45.219 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.407+0000 7f2e117fa700 1 -- 172.21.15.83:0/2555603351 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e0801f030 con 0x7f2e14109100 2024-04-25T11:34:45.219 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.603+0000 7f2e137fe700 1 -- 172.21.15.83:0/2555603351 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f2dfc0395b0 msgr2=0x7f2dfc03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1639787889 2024-04-25T11:34:45.219 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.603+0000 7f2e137fe700 1 --2- 172.21.15.83:0/2555603351 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f2dfc0395b0 0x7f2dfc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:34:45.219 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.668+0000 7f2e1a7b8700 1 -- 172.21.15.83:0/2555603351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f2e1419fd10 con 0x7f2e14109100 2024-04-25T11:34:45.219 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.668+0000 7f2e117fa700 1 -- 172.21.15.83:0/2555603351 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f2e08029260 con 0x7f2e14109100 2024-04-25T11:34:45.219 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.670+0000 7f2e1a7b8700 1 -- 172.21.15.83:0/2555603351 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f2dfc0395b0 msgr2=0x7f2dfc03ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:34:45.219 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.670+0000 7f2e1a7b8700 1 --2- 172.21.15.83:0/2555603351 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f2dfc0395b0 0x7f2dfc03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:45.219 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.670+0000 7f2e1a7b8700 1 -- 172.21.15.83:0/2555603351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14109100 msgr2=0x7f2e1419c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:45.219 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.670+0000 7f2e1a7b8700 1 --2- 172.21.15.83:0/2555603351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14109100 0x7f2e1419c410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2e080116e0 tx=0x7f2e08011aa0 comp rx=0 tx=0).stop 2024-04-25T11:34:45.219 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.670+0000 7f2e1a7b8700 1 -- 172.21.15.83:0/2555603351 shutdown_connections 2024-04-25T11:34:45.220 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.670+0000 7f2e1a7b8700 1 --2- 172.21.15.83:0/2555603351 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7f2dfc0395b0 0x7f2dfc03ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:45.220 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.670+0000 7f2e1a7b8700 1 --2- 172.21.15.83:0/2555603351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e14109100 0x7f2e1419c410 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:45.220 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.670+0000 7f2e1a7b8700 1 -- 172.21.15.83:0/2555603351 >> 172.21.15.83:0/2555603351 conn(0x7f2e14102ab0 msgr2=0x7f2e14103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:45.220 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.670+0000 7f2e1a7b8700 1 -- 172.21.15.83:0/2555603351 shutdown_connections 2024-04-25T11:34:45.220 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:44.670+0000 7f2e1a7b8700 1 -- 172.21.15.83:0/2555603351 wait complete. 2024-04-25T11:34:45.220 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for the mgr to restart... 2024-04-25T11:34:45.220 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr epoch 9... 2024-04-25T11:34:45.462 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:45 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:45.363+0000 7f96c015d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:34:45.742 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:45 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:45.662+0000 7f96c015d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:34:45.743 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:45 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:45.754+0000 7f96c015d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:34:46.134 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:45 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:45.954+0000 7f96c015d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:34:46.406 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:46 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:46.145+0000 7f96c015d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:34:47.042 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:46 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:46.756+0000 7f96c015d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:34:47.407 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:47 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:47.178+0000 7f96c015d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:34:47.407 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:47 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:47.275+0000 7f96c015d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:34:47.907 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:47 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:47.480+0000 7f96c015d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:34:48.335 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:48 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:48.018+0000 7f96c015d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:34:48.335 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:48 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:48.109+0000 7f96c015d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:34:48.643 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:48 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:48.347+0000 7f96c015d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:34:48.643 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:48 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:48.477+0000 7f96c015d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:34:48.643 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:48 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:48.571+0000 7f96c015d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:34:48.907 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:48 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:48.687+0000 7f96c015d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:34:49.406 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:48 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:48.988+0000 7f96c015d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:34:49.684 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:49 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:49.595+0000 7f96c015d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:34:49.936 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:34:49 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:34:49.823+0000 7f96c015d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:34:49.937 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:49 smithi083 ceph-mon[29330]: Active manager daemon x restarted 2024-04-25T11:34:49.937 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:49 smithi083 ceph-mon[29330]: Activating manager daemon x 2024-04-25T11:34:49.937 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:49 smithi083 ceph-mon[29330]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T11:34:49.937 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:49 smithi083 ceph-mon[29330]: mgrmap e10: x(active, starting, since 0.0035073s) 2024-04-25T11:34:49.937 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:34:49.937 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T11:34:49.937 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T11:34:49.937 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T11:34:49.937 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T11:34:49.938 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:49 smithi083 ceph-mon[29330]: Manager daemon x is now available 2024-04-25T11:34:49.938 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:34:51.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T11:34:51.025 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T11:34:51.025 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:50 smithi083 ceph-mon[29330]: mgrmap e11: x(active, since 1.00568s) 2024-04-25T11:34:51.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:51 smithi083 ceph-mon[29330]: [25/Apr/2024:11:34:50] ENGINE Bus STARTING 2024-04-25T11:34:51.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:51 smithi083 ceph-mon[29330]: [25/Apr/2024:11:34:50] ENGINE Serving on https://172.21.15.83:7150 2024-04-25T11:34:51.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:51 smithi083 ceph-mon[29330]: [25/Apr/2024:11:34:50] ENGINE Serving on http://172.21.15.83:8765 2024-04-25T11:34:51.874 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:51 smithi083 ceph-mon[29330]: [25/Apr/2024:11:34:50] ENGINE Bus STARTED 2024-04-25T11:34:51.930 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-25T11:34:51.930 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T11:34:51.930 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T11:34:51.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-25T11:34:51.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.645+0000 7fb6bd494700 1 Processor -- start 2024-04-25T11:34:51.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.645+0000 7fb6bd494700 1 -- start start 2024-04-25T11:34:51.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.645+0000 7fb6bd494700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b8108b30 0x7fb6b8108f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.645+0000 7fb6bd494700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b8109520 con 0x7fb6b8108b30 2024-04-25T11:34:51.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.646+0000 7fb6b6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b8108b30 0x7fb6b8108f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:51.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.646+0000 7fb6b6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b8108b30 0x7fb6b8108f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35822/0 (socket says 172.21.15.83:35822) 2024-04-25T11:34:51.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.646+0000 7fb6b6ffd700 1 -- 172.21.15.83:0/3947603094 learned_addr learned my addr 172.21.15.83:0/3947603094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:51.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.646+0000 7fb6b6ffd700 1 -- 172.21.15.83:0/3947603094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b8106630 con 0x7fb6b8108b30 2024-04-25T11:34:51.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.647+0000 7fb6b6ffd700 1 --2- 172.21.15.83:0/3947603094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b8108b30 0x7fb6b8108f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80098a0 tx=0x7fb6a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bcef2f95053cea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:51.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.647+0000 7fb6b5ffb700 1 -- 172.21.15.83:0/3947603094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a8004160 con 0x7fb6b8108b30 2024-04-25T11:34:51.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.647+0000 7fb6b5ffb700 1 -- 172.21.15.83:0/3947603094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6a8004300 con 0x7fb6b8108b30 2024-04-25T11:34:51.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.647+0000 7fb6b5ffb700 1 -- 172.21.15.83:0/3947603094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a8011a20 con 0x7fb6b8108b30 2024-04-25T11:34:51.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.648+0000 7fb6bd494700 1 -- 172.21.15.83:0/3947603094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b8108b30 msgr2=0x7fb6b8108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:51.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.648+0000 7fb6bd494700 1 --2- 172.21.15.83:0/3947603094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b8108b30 0x7fb6b8108f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80098a0 tx=0x7fb6a8009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.648+0000 7fb6bd494700 1 -- 172.21.15.83:0/3947603094 shutdown_connections 2024-04-25T11:34:51.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.648+0000 7fb6bd494700 1 --2- 172.21.15.83:0/3947603094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b8108b30 0x7fb6b8108f50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.648+0000 7fb6bd494700 1 -- 172.21.15.83:0/3947603094 >> 172.21.15.83:0/3947603094 conn(0x7fb6b81021d0 msgr2=0x7fb6b8104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:51.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.648+0000 7fb6bd494700 1 -- 172.21.15.83:0/3947603094 shutdown_connections 2024-04-25T11:34:51.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.648+0000 7fb6bd494700 1 -- 172.21.15.83:0/3947603094 wait complete. 2024-04-25T11:34:51.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.648+0000 7fb6bd494700 1 Processor -- start 2024-04-25T11:34:51.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.649+0000 7fb6bd494700 1 -- start start 2024-04-25T11:34:51.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.649+0000 7fb6bd494700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b8108b30 0x7fb6b819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.932 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.649+0000 7fb6bd494700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b8107690 con 0x7fb6b8108b30 2024-04-25T11:34:51.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.649+0000 7fb6b6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b8108b30 0x7fb6b819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:51.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.649+0000 7fb6b6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b8108b30 0x7fb6b819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35834/0 (socket says 172.21.15.83:35834) 2024-04-25T11:34:51.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.649+0000 7fb6b6ffd700 1 -- 172.21.15.83:0/919813420 learned_addr learned my addr 172.21.15.83:0/919813420 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:51.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.649+0000 7fb6b6ffd700 1 -- 172.21.15.83:0/919813420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b819ebe0 con 0x7fb6b8108b30 2024-04-25T11:34:51.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.649+0000 7fb6b6ffd700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b8108b30 0x7fb6b819e6a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8004660 tx=0x7fb6a8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:51.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.650+0000 7fb6affff700 1 -- 172.21.15.83:0/919813420 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a8003aa0 con 0x7fb6b8108b30 2024-04-25T11:34:51.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.650+0000 7fb6affff700 1 -- 172.21.15.83:0/919813420 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6a801b8f0 con 0x7fb6b8108b30 2024-04-25T11:34:51.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.650+0000 7fb6bd494700 1 -- 172.21.15.83:0/919813420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b819ee70 con 0x7fb6b8108b30 2024-04-25T11:34:51.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.650+0000 7fb6affff700 1 -- 172.21.15.83:0/919813420 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a8023900 con 0x7fb6b8108b30 2024-04-25T11:34:51.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.650+0000 7fb6bd494700 1 -- 172.21.15.83:0/919813420 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b81a1b60 con 0x7fb6b8108b30 2024-04-25T11:34:51.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.651+0000 7fb6affff700 1 -- 172.21.15.83:0/919813420 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fb6a8023bc0 con 0x7fb6b8108b30 2024-04-25T11:34:51.933 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.651+0000 7fb6affff700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fb6a4039600 0x7fb6a403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.651+0000 7fb6affff700 1 -- 172.21.15.83:0/919813420 --> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb6a403c1d0 con 0x7fb6a4039600 2024-04-25T11:34:51.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.651+0000 7fb6b67fc700 1 -- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fb6a4039600 msgr2=0x7fb6a403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1639787889 2024-04-25T11:34:51.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.651+0000 7fb6affff700 1 -- 172.21.15.83:0/919813420 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb6a804e000 con 0x7fb6b8108b30 2024-04-25T11:34:51.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.651+0000 7fb6b67fc700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fb6a4039600 0x7fb6a403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:34:51.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.852+0000 7fb6b67fc700 1 -- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fb6a4039600 msgr2=0x7fb6a403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1639787889 2024-04-25T11:34:51.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:45.852+0000 7fb6b67fc700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fb6a4039600 0x7fb6a403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:34:51.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:46.253+0000 7fb6b67fc700 1 -- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fb6a4039600 msgr2=0x7fb6a403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1639787889 2024-04-25T11:34:51.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:46.253+0000 7fb6b67fc700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fb6a4039600 0x7fb6a403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:34:51.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:47.054+0000 7fb6b67fc700 1 -- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fb6a4039600 msgr2=0x7fb6a403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1639787889 2024-04-25T11:34:51.934 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:47.054+0000 7fb6b67fc700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fb6a4039600 0x7fb6a403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T11:34:51.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:48.655+0000 7fb6b67fc700 1 -- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fb6a4039600 msgr2=0x7fb6a403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1639787889 2024-04-25T11:34:51.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:48.655+0000 7fb6b67fc700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fb6a4039600 0x7fb6a403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T11:34:51.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:49.827+0000 7fb6affff700 1 -- 172.21.15.83:0/919813420 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7fb6a801baa0 con 0x7fb6b8108b30 2024-04-25T11:34:51.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:49.827+0000 7fb6affff700 1 -- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fb6a4039600 msgr2=0x7fb6a403bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:34:51.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:49.827+0000 7fb6affff700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fb6a4039600 0x7fb6a403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.828+0000 7fb6affff700 1 -- 172.21.15.83:0/919813420 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fb6a8050160 con 0x7fb6b8108b30 2024-04-25T11:34:51.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.829+0000 7fb6affff700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb6a403cfe0 0x7fb6a403f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:51.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.829+0000 7fb6affff700 1 -- 172.21.15.83:0/919813420 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb6a403c1d0 con 0x7fb6a403cfe0 2024-04-25T11:34:51.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.829+0000 7fb6b67fc700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb6a403cfe0 0x7fb6a403f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:51.935 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.830+0000 7fb6b67fc700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb6a403cfe0 0x7fb6a403f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0003a10 tx=0x7fb6a00092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:51.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.830+0000 7fb6affff700 1 -- 172.21.15.83:0/919813420 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fb6a403c1d0 con 0x7fb6a403cfe0 2024-04-25T11:34:51.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.837+0000 7fb6bd494700 1 -- 172.21.15.83:0/919813420 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb6b819f350 con 0x7fb6a403cfe0 2024-04-25T11:34:51.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.837+0000 7fb6affff700 1 -- 172.21.15.83:0/919813420 <== mgr.14152 v2:172.21.15.83:6800/1157946519 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fb6b819f350 con 0x7fb6a403cfe0 2024-04-25T11:34:51.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.837+0000 7fb6bd494700 1 -- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb6a403cfe0 msgr2=0x7fb6a403f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:51.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.837+0000 7fb6bd494700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb6a403cfe0 0x7fb6a403f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0003a10 tx=0x7fb6a00092b0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.837+0000 7fb6bd494700 1 -- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b8108b30 msgr2=0x7fb6b819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:51.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.837+0000 7fb6bd494700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b8108b30 0x7fb6b819e6a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8004660 tx=0x7fb6a8011aa0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.837+0000 7fb6bd494700 1 -- 172.21.15.83:0/919813420 shutdown_connections 2024-04-25T11:34:51.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.837+0000 7fb6bd494700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb6a403cfe0 0x7fb6a403f3d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.837+0000 7fb6bd494700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6b8108b30 0x7fb6b819e6a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.837+0000 7fb6bd494700 1 --2- 172.21.15.83:0/919813420 >> [v2:172.21.15.83:6800/1639787889,v1:172.21.15.83:6801/1639787889] conn(0x7fb6a4039600 0x7fb6a403bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:51.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.837+0000 7fb6bd494700 1 -- 172.21.15.83:0/919813420 >> 172.21.15.83:0/919813420 conn(0x7fb6b81021d0 msgr2=0x7fb6b81952d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:51.936 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.838+0000 7fb6bd494700 1 -- 172.21.15.83:0/919813420 shutdown_connections 2024-04-25T11:34:51.937 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:50.838+0000 7fb6bd494700 1 -- 172.21.15.83:0/919813420 wait complete. 2024-04-25T11:34:51.937 INFO:teuthology.orchestra.run.smithi083.stdout:mgr epoch 9 is available 2024-04-25T11:34:51.937 INFO:teuthology.orchestra.run.smithi083.stdout:Generating a dashboard self-signed certificate... 2024-04-25T11:34:52.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:51 smithi083 ceph-mon[29330]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T11:34:52.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:51 smithi083 ceph-mon[29330]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T11:34:52.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:51 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:34:53.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T11:34:53.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.399+0000 7efd2f399700 1 Processor -- start 2024-04-25T11:34:53.288 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.400+0000 7efd2f399700 1 -- start start 2024-04-25T11:34:53.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.400+0000 7efd2f399700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd281074e0 0x7efd28107900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:53.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.400+0000 7efd2f399700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd28107ed0 con 0x7efd281074e0 2024-04-25T11:34:53.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.400+0000 7efd2d135700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd281074e0 0x7efd28107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:53.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.400+0000 7efd2d135700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd281074e0 0x7efd28107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35886/0 (socket says 172.21.15.83:35886) 2024-04-25T11:34:53.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.400+0000 7efd2d135700 1 -- 172.21.15.83:0/2527429105 learned_addr learned my addr 172.21.15.83:0/2527429105 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:53.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.400+0000 7efd2d135700 1 -- 172.21.15.83:0/2527429105 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd281086e0 con 0x7efd281074e0 2024-04-25T11:34:53.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.401+0000 7efd2d135700 1 --2- 172.21.15.83:0/2527429105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd281074e0 0x7efd28107900 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd2400bb40 tx=0x7efd2400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108c32dc068d9a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:53.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.401+0000 7efd1ffff700 1 -- 172.21.15.83:0/2527429105 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd2400dbe0 con 0x7efd281074e0 2024-04-25T11:34:53.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.401+0000 7efd1ffff700 1 -- 172.21.15.83:0/2527429105 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd240044e0 con 0x7efd281074e0 2024-04-25T11:34:53.289 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.401+0000 7efd1ffff700 1 -- 172.21.15.83:0/2527429105 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd24010470 con 0x7efd281074e0 2024-04-25T11:34:53.290 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.401+0000 7efd2f399700 1 -- 172.21.15.83:0/2527429105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd281074e0 msgr2=0x7efd28107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:53.290 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.401+0000 7efd2f399700 1 --2- 172.21.15.83:0/2527429105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd281074e0 0x7efd28107900 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd2400bb40 tx=0x7efd2400be50 comp rx=0 tx=0).stop 2024-04-25T11:34:53.290 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.401+0000 7efd2f399700 1 -- 172.21.15.83:0/2527429105 shutdown_connections 2024-04-25T11:34:53.290 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.401+0000 7efd2f399700 1 --2- 172.21.15.83:0/2527429105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd281074e0 0x7efd28107900 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.290 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.401+0000 7efd2f399700 1 -- 172.21.15.83:0/2527429105 >> 172.21.15.83:0/2527429105 conn(0x7efd28102b00 msgr2=0x7efd28104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:53.291 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.402+0000 7efd2f399700 1 -- 172.21.15.83:0/2527429105 shutdown_connections 2024-04-25T11:34:53.291 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.402+0000 7efd2f399700 1 -- 172.21.15.83:0/2527429105 wait complete. 2024-04-25T11:34:53.291 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.402+0000 7efd2f399700 1 Processor -- start 2024-04-25T11:34:53.291 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.402+0000 7efd2f399700 1 -- start start 2024-04-25T11:34:53.291 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.402+0000 7efd2f399700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd281074e0 0x7efd2819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:53.291 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.402+0000 7efd2f399700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd2807afb0 con 0x7efd281074e0 2024-04-25T11:34:53.291 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.402+0000 7efd2d135700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd281074e0 0x7efd2819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.402+0000 7efd2d135700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd281074e0 0x7efd2819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35902/0 (socket says 172.21.15.83:35902) 2024-04-25T11:34:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.402+0000 7efd2d135700 1 -- 172.21.15.83:0/2022529089 learned_addr learned my addr 172.21.15.83:0/2022529089 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.403+0000 7efd2d135700 1 -- 172.21.15.83:0/2022529089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2819a750 con 0x7efd281074e0 2024-04-25T11:34:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.403+0000 7efd2d135700 1 --2- 172.21.15.83:0/2022529089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd281074e0 0x7efd2819a210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efd2400d650 tx=0x7efd24003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.403+0000 7efd1e7fc700 1 -- 172.21.15.83:0/2022529089 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd24027070 con 0x7efd281074e0 2024-04-25T11:34:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.403+0000 7efd2f399700 1 -- 172.21.15.83:0/2022529089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2819a9e0 con 0x7efd281074e0 2024-04-25T11:34:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.403+0000 7efd1e7fc700 1 -- 172.21.15.83:0/2022529089 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd24010c20 con 0x7efd281074e0 2024-04-25T11:34:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.403+0000 7efd2f399700 1 -- 172.21.15.83:0/2022529089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2819d6d0 con 0x7efd281074e0 2024-04-25T11:34:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.404+0000 7efd1e7fc700 1 -- 172.21.15.83:0/2022529089 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd24022b10 con 0x7efd281074e0 2024-04-25T11:34:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.404+0000 7efd1e7fc700 1 -- 172.21.15.83:0/2022529089 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7efd2402a070 con 0x7efd281074e0 2024-04-25T11:34:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.404+0000 7efd2f399700 1 -- 172.21.15.83:0/2022529089 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd280640a0 con 0x7efd281074e0 2024-04-25T11:34:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.404+0000 7efd1e7fc700 1 --2- 172.21.15.83:0/2022529089 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efd140395b0 0x7efd1403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.404+0000 7efd1e7fc700 1 -- 172.21.15.83:0/2022529089 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efd2401f070 con 0x7efd281074e0 2024-04-25T11:34:53.292 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.405+0000 7efd2c934700 1 --2- 172.21.15.83:0/2022529089 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efd140395b0 0x7efd1403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:53.293 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.405+0000 7efd2c934700 1 --2- 172.21.15.83:0/2022529089 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efd140395b0 0x7efd1403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efd18006fd0 tx=0x7efd18006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:53.293 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.410+0000 7efd1e7fc700 1 -- 172.21.15.83:0/2022529089 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd24015070 con 0x7efd281074e0 2024-04-25T11:34:53.293 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.630+0000 7efd2f399700 1 -- 172.21.15.83:0/2022529089 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7efd2802e1d0 con 0x7efd140395b0 2024-04-25T11:34:53.293 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.746+0000 7efd1e7fc700 1 -- 172.21.15.83:0/2022529089 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7efd2802e1d0 con 0x7efd140395b0 2024-04-25T11:34:53.293 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.748+0000 7efd2f399700 1 -- 172.21.15.83:0/2022529089 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efd140395b0 msgr2=0x7efd1403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:53.293 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.748+0000 7efd2f399700 1 --2- 172.21.15.83:0/2022529089 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efd140395b0 0x7efd1403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efd18006fd0 tx=0x7efd18006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:53.293 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.748+0000 7efd2f399700 1 -- 172.21.15.83:0/2022529089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd281074e0 msgr2=0x7efd2819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:53.293 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.748+0000 7efd2f399700 1 --2- 172.21.15.83:0/2022529089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd281074e0 0x7efd2819a210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efd2400d650 tx=0x7efd24003ff0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.293 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.748+0000 7efd2f399700 1 -- 172.21.15.83:0/2022529089 shutdown_connections 2024-04-25T11:34:53.293 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.748+0000 7efd2f399700 1 --2- 172.21.15.83:0/2022529089 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efd140395b0 0x7efd1403ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.293 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.748+0000 7efd2f399700 1 --2- 172.21.15.83:0/2022529089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd281074e0 0x7efd2819a210 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:53.293 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.748+0000 7efd2f399700 1 -- 172.21.15.83:0/2022529089 >> 172.21.15.83:0/2022529089 conn(0x7efd28102b00 msgr2=0x7efd28190eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:53.294 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.748+0000 7efd2f399700 1 -- 172.21.15.83:0/2022529089 shutdown_connections 2024-04-25T11:34:53.294 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:52.748+0000 7efd2f399700 1 -- 172.21.15.83:0/2022529089 wait complete. 2024-04-25T11:34:53.294 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial admin user... 2024-04-25T11:34:53.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:53 smithi083 ceph-mon[29330]: mgrmap e12: x(active, since 2s) 2024-04-25T11:34:53.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:53 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:34:53.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:53 smithi083 ceph-mon[29330]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:34:53.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:53 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:34:53.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:53 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:34:54.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:54 smithi083 ceph-mon[29330]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:34:54.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:34:55.387 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$t7Glf4/OKl/sr8K4UPcOQu6ZT5l86tXCkqc3Uxws8AI2xU4mAyD6y", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714044894, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T11:34:55.387 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.722+0000 7f1cd524e700 1 Processor -- start 2024-04-25T11:34:55.387 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.722+0000 7f1cd524e700 1 -- start start 2024-04-25T11:34:55.387 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.723+0000 7f1cd524e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd0108b40 0x7f1cd0108f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:55.388 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.723+0000 7f1cd524e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0109530 con 0x7f1cd0108b40 2024-04-25T11:34:55.388 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.723+0000 7f1cced9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd0108b40 0x7f1cd0108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:55.388 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.723+0000 7f1cced9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd0108b40 0x7f1cd0108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53588/0 (socket says 172.21.15.83:53588) 2024-04-25T11:34:55.388 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.723+0000 7f1cced9d700 1 -- 172.21.15.83:0/1686171533 learned_addr learned my addr 172.21.15.83:0/1686171533 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:55.388 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.723+0000 7f1cced9d700 1 -- 172.21.15.83:0/1686171533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd0107710 con 0x7f1cd0108b40 2024-04-25T11:34:55.388 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.723+0000 7f1cced9d700 1 --2- 172.21.15.83:0/1686171533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd0108b40 0x7f1cd0108f60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40098a0 tx=0x7f1cc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babafc8429479888 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:55.388 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.723+0000 7f1ccdd9b700 1 -- 172.21.15.83:0/1686171533 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cc4004160 con 0x7f1cd0108b40 2024-04-25T11:34:55.388 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.723+0000 7f1ccdd9b700 1 -- 172.21.15.83:0/1686171533 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cc4004300 con 0x7f1cd0108b40 2024-04-25T11:34:55.388 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.723+0000 7f1ccdd9b700 1 -- 172.21.15.83:0/1686171533 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cc4011a20 con 0x7f1cd0108b40 2024-04-25T11:34:55.388 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.724+0000 7f1cd524e700 1 -- 172.21.15.83:0/1686171533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd0108b40 msgr2=0x7f1cd0108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:55.388 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.724+0000 7f1cd524e700 1 --2- 172.21.15.83:0/1686171533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd0108b40 0x7f1cd0108f60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40098a0 tx=0x7f1cc4009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.389 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.724+0000 7f1cd524e700 1 -- 172.21.15.83:0/1686171533 shutdown_connections 2024-04-25T11:34:55.389 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.724+0000 7f1cd524e700 1 --2- 172.21.15.83:0/1686171533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd0108b40 0x7f1cd0108f60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.389 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.724+0000 7f1cd524e700 1 -- 172.21.15.83:0/1686171533 >> 172.21.15.83:0/1686171533 conn(0x7f1cd0102200 msgr2=0x7f1cd0104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:55.389 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.724+0000 7f1cd524e700 1 -- 172.21.15.83:0/1686171533 shutdown_connections 2024-04-25T11:34:55.389 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.724+0000 7f1cd524e700 1 -- 172.21.15.83:0/1686171533 wait complete. 2024-04-25T11:34:55.389 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.725+0000 7f1cd524e700 1 Processor -- start 2024-04-25T11:34:55.389 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.725+0000 7f1cd524e700 1 -- start start 2024-04-25T11:34:55.389 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.725+0000 7f1cd524e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd0108b40 0x7f1cd0107080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:55.389 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.725+0000 7f1cd524e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd010bad0 con 0x7f1cd0108b40 2024-04-25T11:34:55.389 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.725+0000 7f1cced9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd0108b40 0x7f1cd0107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:55.389 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.725+0000 7f1cced9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd0108b40 0x7f1cd0107080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53594/0 (socket says 172.21.15.83:53594) 2024-04-25T11:34:55.389 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.725+0000 7f1cced9d700 1 -- 172.21.15.83:0/1482905091 learned_addr learned my addr 172.21.15.83:0/1482905091 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:55.390 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.725+0000 7f1cced9d700 1 -- 172.21.15.83:0/1482905091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd019f1a0 con 0x7f1cd0108b40 2024-04-25T11:34:55.390 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.726+0000 7f1cced9d700 1 --2- 172.21.15.83:0/1482905091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd0108b40 0x7f1cd0107080 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40116e0 tx=0x7f1cc4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:55.390 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.726+0000 7f1cb7fff700 1 -- 172.21.15.83:0/1482905091 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cc40038b0 con 0x7f1cd0108b40 2024-04-25T11:34:55.390 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.726+0000 7f1cb7fff700 1 -- 172.21.15.83:0/1482905091 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cc4003a50 con 0x7f1cd0108b40 2024-04-25T11:34:55.390 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.726+0000 7f1cd524e700 1 -- 172.21.15.83:0/1482905091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd019f370 con 0x7f1cd0108b40 2024-04-25T11:34:55.390 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.726+0000 7f1cb7fff700 1 -- 172.21.15.83:0/1482905091 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cc4019760 con 0x7f1cd0108b40 2024-04-25T11:34:55.390 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.726+0000 7f1cd524e700 1 -- 172.21.15.83:0/1482905091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd019f710 con 0x7f1cd0108b40 2024-04-25T11:34:55.390 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.727+0000 7f1cb7fff700 1 -- 172.21.15.83:0/1482905091 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f1cc4019a60 con 0x7f1cd0108b40 2024-04-25T11:34:55.390 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.727+0000 7f1cd524e700 1 -- 172.21.15.83:0/1482905091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd00640a0 con 0x7f1cd0108b40 2024-04-25T11:34:55.390 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.727+0000 7f1cb7fff700 1 --2- 172.21.15.83:0/1482905091 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1cb80395b0 0x7f1cb803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:55.390 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.727+0000 7f1cb7fff700 1 -- 172.21.15.83:0/1482905091 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1cc404deb0 con 0x7f1cd0108b40 2024-04-25T11:34:55.391 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.727+0000 7f1cce59c700 1 --2- 172.21.15.83:0/1482905091 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1cb80395b0 0x7f1cb803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:55.391 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.727+0000 7f1cce59c700 1 --2- 172.21.15.83:0/1482905091 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1cb80395b0 0x7f1cb803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0006fd0 tx=0x7f1cc0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:55.391 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.732+0000 7f1cb7fff700 1 -- 172.21.15.83:0/1482905091 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1cc4016dd0 con 0x7f1cd0108b40 2024-04-25T11:34:55.391 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:53.970+0000 7f1cd524e700 1 -- 172.21.15.83:0/1482905091 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f1cd019fa50 con 0x7f1cb80395b0 2024-04-25T11:34:55.391 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:54.194+0000 7f1cb7fff700 1 -- 172.21.15.83:0/1482905091 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f1cd019fa50 con 0x7f1cb80395b0 2024-04-25T11:34:55.391 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:54.196+0000 7f1cd524e700 1 -- 172.21.15.83:0/1482905091 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1cb80395b0 msgr2=0x7f1cb803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:55.391 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:54.196+0000 7f1cd524e700 1 --2- 172.21.15.83:0/1482905091 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1cb80395b0 0x7f1cb803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0006fd0 tx=0x7f1cc0006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:55.391 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:54.196+0000 7f1cd524e700 1 -- 172.21.15.83:0/1482905091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd0108b40 msgr2=0x7f1cd0107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:55.391 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:54.196+0000 7f1cd524e700 1 --2- 172.21.15.83:0/1482905091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd0108b40 0x7f1cd0107080 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40116e0 tx=0x7f1cc4011aa0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.391 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:54.196+0000 7f1cd524e700 1 -- 172.21.15.83:0/1482905091 shutdown_connections 2024-04-25T11:34:55.391 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:54.196+0000 7f1cd524e700 1 --2- 172.21.15.83:0/1482905091 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1cb80395b0 0x7f1cb803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.391 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:54.196+0000 7f1cd524e700 1 --2- 172.21.15.83:0/1482905091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cd0108b40 0x7f1cd0107080 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:55.391 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:54.196+0000 7f1cd524e700 1 -- 172.21.15.83:0/1482905091 >> 172.21.15.83:0/1482905091 conn(0x7f1cd0102200 msgr2=0x7f1cd0102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:55.392 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:54.197+0000 7f1cd524e700 1 -- 172.21.15.83:0/1482905091 shutdown_connections 2024-04-25T11:34:55.392 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:54.197+0000 7f1cd524e700 1 -- 172.21.15.83:0/1482905091 wait complete. 2024-04-25T11:34:55.392 INFO:teuthology.orchestra.run.smithi083.stdout:Fetching dashboard port number... 2024-04-25T11:34:56.602 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T11:34:56.602 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.828+0000 7f37a2dfa700 1 Processor -- start 2024-04-25T11:34:56.602 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.828+0000 7f37a2dfa700 1 -- start start 2024-04-25T11:34:56.602 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.828+0000 7f37a2dfa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379c1090a0 0x7f379c10b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:56.602 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.828+0000 7f37a2dfa700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f379c07be50 con 0x7f379c1090a0 2024-04-25T11:34:56.602 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.828+0000 7f37a0b96700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379c1090a0 0x7f379c10b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:56.602 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.828+0000 7f37a0b96700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379c1090a0 0x7f379c10b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53610/0 (socket says 172.21.15.83:53610) 2024-04-25T11:34:56.602 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.828+0000 7f37a0b96700 1 -- 172.21.15.83:0/733964573 learned_addr learned my addr 172.21.15.83:0/733964573 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:56.602 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.828+0000 7f37a0b96700 1 -- 172.21.15.83:0/733964573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c10b9d0 con 0x7f379c1090a0 2024-04-25T11:34:56.602 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.828+0000 7f37a0b96700 1 --2- 172.21.15.83:0/733964573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379c1090a0 0x7f379c10b490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37900098a0 tx=0x7f3790009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eaed1345396b7cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:56.602 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.829+0000 7f379b7fe700 1 -- 172.21.15.83:0/733964573 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3790004160 con 0x7f379c1090a0 2024-04-25T11:34:56.602 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.829+0000 7f379b7fe700 1 -- 172.21.15.83:0/733964573 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3790004300 con 0x7f379c1090a0 2024-04-25T11:34:56.603 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.829+0000 7f379b7fe700 1 -- 172.21.15.83:0/733964573 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3790011a20 con 0x7f379c1090a0 2024-04-25T11:34:56.603 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.829+0000 7f37a2dfa700 1 -- 172.21.15.83:0/733964573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379c1090a0 msgr2=0x7f379c10b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:56.603 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.829+0000 7f37a2dfa700 1 --2- 172.21.15.83:0/733964573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379c1090a0 0x7f379c10b490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37900098a0 tx=0x7f3790009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.603 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.829+0000 7f37a2dfa700 1 -- 172.21.15.83:0/733964573 shutdown_connections 2024-04-25T11:34:56.603 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.829+0000 7f37a2dfa700 1 --2- 172.21.15.83:0/733964573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379c1090a0 0x7f379c10b490 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.603 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.829+0000 7f37a2dfa700 1 -- 172.21.15.83:0/733964573 >> 172.21.15.83:0/733964573 conn(0x7f379c102a70 msgr2=0x7f379c104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:56.603 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.829+0000 7f37a2dfa700 1 -- 172.21.15.83:0/733964573 shutdown_connections 2024-04-25T11:34:56.603 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.829+0000 7f37a2dfa700 1 -- 172.21.15.83:0/733964573 wait complete. 2024-04-25T11:34:56.603 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.830+0000 7f37a2dfa700 1 Processor -- start 2024-04-25T11:34:56.603 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.830+0000 7f37a2dfa700 1 -- start start 2024-04-25T11:34:56.603 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.830+0000 7f37a2dfa700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379c1090a0 0x7f379c19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:56.603 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.830+0000 7f37a2dfa700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f379c10c9d0 con 0x7f379c1090a0 2024-04-25T11:34:56.603 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.830+0000 7f37a0b96700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379c1090a0 0x7f379c19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:56.604 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.830+0000 7f37a0b96700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379c1090a0 0x7f379c19c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53620/0 (socket says 172.21.15.83:53620) 2024-04-25T11:34:56.604 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.830+0000 7f37a0b96700 1 -- 172.21.15.83:0/2946707777 learned_addr learned my addr 172.21.15.83:0/2946707777 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:56.604 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.831+0000 7f37a0b96700 1 -- 172.21.15.83:0/2946707777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379c19c970 con 0x7f379c1090a0 2024-04-25T11:34:56.604 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.831+0000 7f37a0b96700 1 --2- 172.21.15.83:0/2946707777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379c1090a0 0x7f379c19c430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f37900116e0 tx=0x7f3790011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:56.604 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.831+0000 7f3799ffb700 1 -- 172.21.15.83:0/2946707777 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3790003a60 con 0x7f379c1090a0 2024-04-25T11:34:56.604 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.831+0000 7f3799ffb700 1 -- 172.21.15.83:0/2946707777 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f379001b8f0 con 0x7f379c1090a0 2024-04-25T11:34:56.604 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.831+0000 7f3799ffb700 1 -- 172.21.15.83:0/2946707777 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3790019860 con 0x7f379c1090a0 2024-04-25T11:34:56.604 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.831+0000 7f37a2dfa700 1 -- 172.21.15.83:0/2946707777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379c19cc00 con 0x7f379c1090a0 2024-04-25T11:34:56.604 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.831+0000 7f37a2dfa700 1 -- 172.21.15.83:0/2946707777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f379c19f8f0 con 0x7f379c1090a0 2024-04-25T11:34:56.604 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.832+0000 7f3799ffb700 1 -- 172.21.15.83:0/2946707777 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f379001b440 con 0x7f379c1090a0 2024-04-25T11:34:56.604 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.832+0000 7f37a2dfa700 1 -- 172.21.15.83:0/2946707777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379c0640a0 con 0x7f379c1090a0 2024-04-25T11:34:56.604 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.832+0000 7f3799ffb700 1 --2- 172.21.15.83:0/2946707777 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f378c039610 0x7f378c03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:56.604 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.832+0000 7f3799ffb700 1 -- 172.21.15.83:0/2946707777 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f379004ddd0 con 0x7f379c1090a0 2024-04-25T11:34:56.604 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.832+0000 7f379bfff700 1 --2- 172.21.15.83:0/2946707777 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f378c039610 0x7f378c03bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:56.605 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.833+0000 7f379bfff700 1 --2- 172.21.15.83:0/2946707777 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f378c039610 0x7f378c03bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3788006fd0 tx=0x7f3788006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:56.605 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.837+0000 7f3799ffb700 1 -- 172.21.15.83:0/2946707777 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3790051030 con 0x7f379c1090a0 2024-04-25T11:34:56.605 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:55.879+0000 7f3799ffb700 1 -- 172.21.15.83:0/2946707777 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f379000fb40 con 0x7f379c1090a0 2024-04-25T11:34:56.605 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:56.035+0000 7f37a2dfa700 1 -- 172.21.15.83:0/2946707777 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f379c19fac0 con 0x7f379c1090a0 2024-04-25T11:34:56.605 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:56.035+0000 7f3799ffb700 1 -- 172.21.15.83:0/2946707777 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f379001f030 con 0x7f379c1090a0 2024-04-25T11:34:56.605 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:56.037+0000 7f37a2dfa700 1 -- 172.21.15.83:0/2946707777 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f378c039610 msgr2=0x7f378c03bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:56.605 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:56.037+0000 7f37a2dfa700 1 --2- 172.21.15.83:0/2946707777 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f378c039610 0x7f378c03bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3788006fd0 tx=0x7f3788006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:56.605 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:56.037+0000 7f37a2dfa700 1 -- 172.21.15.83:0/2946707777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379c1090a0 msgr2=0x7f379c19c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:56.605 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:56.037+0000 7f37a2dfa700 1 --2- 172.21.15.83:0/2946707777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379c1090a0 0x7f379c19c430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f37900116e0 tx=0x7f3790011aa0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.605 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:56.037+0000 7f37a2dfa700 1 -- 172.21.15.83:0/2946707777 shutdown_connections 2024-04-25T11:34:56.605 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:56.037+0000 7f37a2dfa700 1 --2- 172.21.15.83:0/2946707777 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f378c039610 0x7f378c03bad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.605 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:56.037+0000 7f37a2dfa700 1 --2- 172.21.15.83:0/2946707777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379c1090a0 0x7f379c19c430 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:56.605 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:56.037+0000 7f37a2dfa700 1 -- 172.21.15.83:0/2946707777 >> 172.21.15.83:0/2946707777 conn(0x7f379c102a70 msgr2=0x7f379c103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:56.605 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:56.037+0000 7f37a2dfa700 1 -- 172.21.15.83:0/2946707777 shutdown_connections 2024-04-25T11:34:56.606 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:56.037+0000 7f37a2dfa700 1 -- 172.21.15.83:0/2946707777 wait complete. 2024-04-25T11:34:56.608 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T11:34:56.608 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-04-25T11:34:56.615 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T11:34:56.615 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-25T11:34:56.615 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-04-25T11:34:56.615 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T11:34:56.616 INFO:teuthology.orchestra.run.smithi083.stdout:Ceph Dashboard is now available at: 2024-04-25T11:34:56.617 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:34:56.617 INFO:teuthology.orchestra.run.smithi083.stdout: URL: https://smithi083:8443/ 2024-04-25T11:34:56.617 INFO:teuthology.orchestra.run.smithi083.stdout: User: admin 2024-04-25T11:34:56.617 INFO:teuthology.orchestra.run.smithi083.stdout: Password: 0wrmay5udq 2024-04-25T11:34:56.617 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:34:56.617 INFO:teuthology.orchestra.run.smithi083.stdout:Saving cluster configuration to /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config directory 2024-04-25T11:34:56.618 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling autotune for osd_memory_target 2024-04-25T11:34:57.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:56 smithi083 ceph-mon[29330]: mgrmap e13: x(active, since 6s) 2024-04-25T11:34:57.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:56 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2946707777' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T11:34:57.749 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.076+0000 7feab9df2700 1 Processor -- start 2024-04-25T11:34:57.749 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.076+0000 7feab9df2700 1 -- start start 2024-04-25T11:34:57.749 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.076+0000 7feab9df2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab4106e90 0x7feab41072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:57.749 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.076+0000 7feab9df2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feab4063d00 con 0x7feab4106e90 2024-04-25T11:34:57.749 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.076+0000 7feab37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab4106e90 0x7feab41072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:57.750 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.076+0000 7feab37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab4106e90 0x7feab41072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53622/0 (socket says 172.21.15.83:53622) 2024-04-25T11:34:57.750 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.076+0000 7feab37fe700 1 -- 172.21.15.83:0/731096353 learned_addr learned my addr 172.21.15.83:0/731096353 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:57.750 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.076+0000 7feab37fe700 1 -- 172.21.15.83:0/731096353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab41077f0 con 0x7feab4106e90 2024-04-25T11:34:57.750 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.077+0000 7feab37fe700 1 --2- 172.21.15.83:0/731096353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab4106e90 0x7feab41072b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feaa40098a0 tx=0x7feaa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a36d07ae087f391 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:57.750 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.077+0000 7feab27fc700 1 -- 172.21.15.83:0/731096353 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaa4004160 con 0x7feab4106e90 2024-04-25T11:34:57.750 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.077+0000 7feab27fc700 1 -- 172.21.15.83:0/731096353 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaa4004300 con 0x7feab4106e90 2024-04-25T11:34:57.750 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.077+0000 7feab27fc700 1 -- 172.21.15.83:0/731096353 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaa4011a20 con 0x7feab4106e90 2024-04-25T11:34:57.750 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.077+0000 7feab9df2700 1 -- 172.21.15.83:0/731096353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab4106e90 msgr2=0x7feab41072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:57.750 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.077+0000 7feab9df2700 1 --2- 172.21.15.83:0/731096353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab4106e90 0x7feab41072b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feaa40098a0 tx=0x7feaa4009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.750 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.077+0000 7feab9df2700 1 -- 172.21.15.83:0/731096353 shutdown_connections 2024-04-25T11:34:57.750 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.077+0000 7feab9df2700 1 --2- 172.21.15.83:0/731096353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab4106e90 0x7feab41072b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.750 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.077+0000 7feab9df2700 1 -- 172.21.15.83:0/731096353 >> 172.21.15.83:0/731096353 conn(0x7feab4102ab0 msgr2=0x7feab4104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:57.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.077+0000 7feab9df2700 1 -- 172.21.15.83:0/731096353 shutdown_connections 2024-04-25T11:34:57.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.078+0000 7feab9df2700 1 -- 172.21.15.83:0/731096353 wait complete. 2024-04-25T11:34:57.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.078+0000 7feab9df2700 1 Processor -- start 2024-04-25T11:34:57.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.078+0000 7feab9df2700 1 -- start start 2024-04-25T11:34:57.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.078+0000 7feab9df2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab4106e90 0x7feab419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:57.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.078+0000 7feab9df2700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feab4108860 con 0x7feab4106e90 2024-04-25T11:34:57.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.079+0000 7feab37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab4106e90 0x7feab419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:57.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.079+0000 7feab37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab4106e90 0x7feab419e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53624/0 (socket says 172.21.15.83:53624) 2024-04-25T11:34:57.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.079+0000 7feab37fe700 1 -- 172.21.15.83:0/1306901479 learned_addr learned my addr 172.21.15.83:0/1306901479 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:57.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.079+0000 7feab37fe700 1 -- 172.21.15.83:0/1306901479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab419eb50 con 0x7feab4106e90 2024-04-25T11:34:57.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.079+0000 7feab37fe700 1 --2- 172.21.15.83:0/1306901479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab4106e90 0x7feab419e610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feaa4000c00 tx=0x7feaa4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:57.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.079+0000 7feab0ff9700 1 -- 172.21.15.83:0/1306901479 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaa4003aa0 con 0x7feab4106e90 2024-04-25T11:34:57.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.079+0000 7feab0ff9700 1 -- 172.21.15.83:0/1306901479 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaa401b8f0 con 0x7feab4106e90 2024-04-25T11:34:57.751 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.079+0000 7feab9df2700 1 -- 172.21.15.83:0/1306901479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab419ede0 con 0x7feab4106e90 2024-04-25T11:34:57.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.079+0000 7feab0ff9700 1 -- 172.21.15.83:0/1306901479 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feaa4023870 con 0x7feab4106e90 2024-04-25T11:34:57.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.079+0000 7feab9df2700 1 -- 172.21.15.83:0/1306901479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab41a1ad0 con 0x7feab4106e90 2024-04-25T11:34:57.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.080+0000 7feab0ff9700 1 -- 172.21.15.83:0/1306901479 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7feaa4023b20 con 0x7feab4106e90 2024-04-25T11:34:57.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.080+0000 7feab9df2700 1 -- 172.21.15.83:0/1306901479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feab40640a0 con 0x7feab4106e90 2024-04-25T11:34:57.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.080+0000 7feab0ff9700 1 --2- 172.21.15.83:0/1306901479 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fea9c039700 0x7fea9c03bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:57.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.080+0000 7feab0ff9700 1 -- 172.21.15.83:0/1306901479 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feaa404e140 con 0x7feab4106e90 2024-04-25T11:34:57.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.080+0000 7feab2ffd700 1 --2- 172.21.15.83:0/1306901479 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fea9c039700 0x7fea9c03bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:57.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.081+0000 7feab2ffd700 1 --2- 172.21.15.83:0/1306901479 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fea9c039700 0x7fea9c03bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feaa8006fd0 tx=0x7feaa8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:57.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.085+0000 7feab0ff9700 1 -- 172.21.15.83:0/1306901479 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feaa401f030 con 0x7feab4106e90 2024-04-25T11:34:57.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.282+0000 7feab9df2700 1 -- 172.21.15.83:0/1306901479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7feab402e1b0 con 0x7feab4106e90 2024-04-25T11:34:57.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.283+0000 7feab0ff9700 1 -- 172.21.15.83:0/1306901479 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7feaa4023e10 con 0x7feab4106e90 2024-04-25T11:34:57.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.285+0000 7feab9df2700 1 -- 172.21.15.83:0/1306901479 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fea9c039700 msgr2=0x7fea9c03bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:57.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.285+0000 7feab9df2700 1 --2- 172.21.15.83:0/1306901479 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fea9c039700 0x7fea9c03bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feaa8006fd0 tx=0x7feaa8006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:57.752 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.285+0000 7feab9df2700 1 -- 172.21.15.83:0/1306901479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab4106e90 msgr2=0x7feab419e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:57.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.285+0000 7feab9df2700 1 --2- 172.21.15.83:0/1306901479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab4106e90 0x7feab419e610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feaa4000c00 tx=0x7feaa4011aa0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.285+0000 7feab9df2700 1 -- 172.21.15.83:0/1306901479 shutdown_connections 2024-04-25T11:34:57.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.285+0000 7feab9df2700 1 --2- 172.21.15.83:0/1306901479 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fea9c039700 0x7fea9c03bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.285+0000 7feab9df2700 1 --2- 172.21.15.83:0/1306901479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feab4106e90 0x7feab419e610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:57.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.285+0000 7feab9df2700 1 -- 172.21.15.83:0/1306901479 >> 172.21.15.83:0/1306901479 conn(0x7feab4102ab0 msgr2=0x7feab41952b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:57.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.285+0000 7feab9df2700 1 -- 172.21.15.83:0/1306901479 shutdown_connections 2024-04-25T11:34:57.753 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:57.285+0000 7feab9df2700 1 -- 172.21.15.83:0/1306901479 wait complete. 2024-04-25T11:34:59.039 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.173+0000 7fdec1b9f700 1 Processor -- start 2024-04-25T11:34:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.173+0000 7fdec1b9f700 1 -- start start 2024-04-25T11:34:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.173+0000 7fdec1b9f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc108600 0x7fdebc108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.173+0000 7fdec1b9f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc108ff0 con 0x7fdebc108600 2024-04-25T11:34:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.173+0000 7fdebb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc108600 0x7fdebc108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.173+0000 7fdebb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc108600 0x7fdebc108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53636/0 (socket says 172.21.15.83:53636) 2024-04-25T11:34:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.173+0000 7fdebb7fe700 1 -- 172.21.15.83:0/1819345298 learned_addr learned my addr 172.21.15.83:0/1819345298 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.173+0000 7fdebb7fe700 1 -- 172.21.15.83:0/1819345298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc1097b0 con 0x7fdebc108600 2024-04-25T11:34:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.173+0000 7fdebb7fe700 1 --2- 172.21.15.83:0/1819345298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc108600 0x7fdebc108a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0098a0 tx=0x7fdeac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20fba10bcc2614f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.174+0000 7fdeba7fc700 1 -- 172.21.15.83:0/1819345298 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdeac004160 con 0x7fdebc108600 2024-04-25T11:34:59.040 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.174+0000 7fdeba7fc700 1 -- 172.21.15.83:0/1819345298 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeac004300 con 0x7fdebc108600 2024-04-25T11:34:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.174+0000 7fdeba7fc700 1 -- 172.21.15.83:0/1819345298 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdeac011a20 con 0x7fdebc108600 2024-04-25T11:34:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.174+0000 7fdec1b9f700 1 -- 172.21.15.83:0/1819345298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc108600 msgr2=0x7fdebc108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.174+0000 7fdec1b9f700 1 --2- 172.21.15.83:0/1819345298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc108600 0x7fdebc108a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0098a0 tx=0x7fdeac009bb0 comp rx=0 tx=0).stop 2024-04-25T11:34:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.174+0000 7fdec1b9f700 1 -- 172.21.15.83:0/1819345298 shutdown_connections 2024-04-25T11:34:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.174+0000 7fdec1b9f700 1 --2- 172.21.15.83:0/1819345298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc108600 0x7fdebc108a20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.174+0000 7fdec1b9f700 1 -- 172.21.15.83:0/1819345298 >> 172.21.15.83:0/1819345298 conn(0x7fdebc103bc0 msgr2=0x7fdebc105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.174+0000 7fdec1b9f700 1 -- 172.21.15.83:0/1819345298 shutdown_connections 2024-04-25T11:34:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.174+0000 7fdec1b9f700 1 -- 172.21.15.83:0/1819345298 wait complete. 2024-04-25T11:34:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.175+0000 7fdec1b9f700 1 Processor -- start 2024-04-25T11:34:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.175+0000 7fdec1b9f700 1 -- start start 2024-04-25T11:34:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.175+0000 7fdec1b9f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc19c620 0x7fdebc19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.175+0000 7fdec1b9f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc10cb60 con 0x7fdebc19c620 2024-04-25T11:34:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.175+0000 7fdebb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc19c620 0x7fdebc19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:59.041 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.175+0000 7fdebb7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc19c620 0x7fdebc19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53646/0 (socket says 172.21.15.83:53646) 2024-04-25T11:34:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.175+0000 7fdebb7fe700 1 -- 172.21.15.83:0/3367465031 learned_addr learned my addr 172.21.15.83:0/3367465031 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:34:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.176+0000 7fdebb7fe700 1 -- 172.21.15.83:0/3367465031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc19cf80 con 0x7fdebc19c620 2024-04-25T11:34:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.176+0000 7fdebb7fe700 1 --2- 172.21.15.83:0/3367465031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc19c620 0x7fdebc19ca40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdeac015000 tx=0x7fdeac0117c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.176+0000 7fdeb8ff9700 1 -- 172.21.15.83:0/3367465031 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdeac011ad0 con 0x7fdebc19c620 2024-04-25T11:34:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.176+0000 7fdec1b9f700 1 -- 172.21.15.83:0/3367465031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdebc19d210 con 0x7fdebc19c620 2024-04-25T11:34:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.176+0000 7fdeb8ff9700 1 -- 172.21.15.83:0/3367465031 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeac003b50 con 0x7fdebc19c620 2024-04-25T11:34:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.176+0000 7fdeb8ff9700 1 -- 172.21.15.83:0/3367465031 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdeac01be40 con 0x7fdebc19c620 2024-04-25T11:34:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.176+0000 7fdec1b9f700 1 -- 172.21.15.83:0/3367465031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdebc19fcf0 con 0x7fdebc19c620 2024-04-25T11:34:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.177+0000 7fdeb8ff9700 1 -- 172.21.15.83:0/3367465031 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdeac01c450 con 0x7fdebc19c620 2024-04-25T11:34:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.177+0000 7fdec1b9f700 1 -- 172.21.15.83:0/3367465031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdebc10cce0 con 0x7fdebc19c620 2024-04-25T11:34:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.177+0000 7fdeb8ff9700 1 --2- 172.21.15.83:0/3367465031 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdea4039650 0x7fdea403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:34:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.177+0000 7fdeb8ff9700 1 -- 172.21.15.83:0/3367465031 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdeac04e490 con 0x7fdebc19c620 2024-04-25T11:34:59.042 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.177+0000 7fdebaffd700 1 --2- 172.21.15.83:0/3367465031 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdea4039650 0x7fdea403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:34:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.177+0000 7fdebaffd700 1 --2- 172.21.15.83:0/3367465031 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdea4039650 0x7fdea403bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0006fd0 tx=0x7fdeb0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:34:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.182+0000 7fdeb8ff9700 1 -- 172.21.15.83:0/3367465031 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdeac00b640 con 0x7fdebc19c620 2024-04-25T11:34:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.491+0000 7fdec1b9f700 1 -- 172.21.15.83:0/3367465031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fdebc1a0510 con 0x7fdebc19c620 2024-04-25T11:34:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.493+0000 7fdeb8ff9700 1 -- 172.21.15.83:0/3367465031 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fdeac00b3e0 con 0x7fdebc19c620 2024-04-25T11:34:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.495+0000 7fdec1b9f700 1 -- 172.21.15.83:0/3367465031 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdea4039650 msgr2=0x7fdea403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.495+0000 7fdec1b9f700 1 --2- 172.21.15.83:0/3367465031 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdea4039650 0x7fdea403bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0006fd0 tx=0x7fdeb0006e40 comp rx=0 tx=0).stop 2024-04-25T11:34:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.495+0000 7fdec1b9f700 1 -- 172.21.15.83:0/3367465031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc19c620 msgr2=0x7fdebc19ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:34:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.495+0000 7fdec1b9f700 1 --2- 172.21.15.83:0/3367465031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc19c620 0x7fdebc19ca40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdeac015000 tx=0x7fdeac0117c0 comp rx=0 tx=0).stop 2024-04-25T11:34:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.495+0000 7fdec1b9f700 1 -- 172.21.15.83:0/3367465031 shutdown_connections 2024-04-25T11:34:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.495+0000 7fdec1b9f700 1 --2- 172.21.15.83:0/3367465031 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdea4039650 0x7fdea403bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.495+0000 7fdec1b9f700 1 --2- 172.21.15.83:0/3367465031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdebc19c620 0x7fdebc19ca40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:34:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.495+0000 7fdec1b9f700 1 -- 172.21.15.83:0/3367465031 >> 172.21.15.83:0/3367465031 conn(0x7fdebc103bc0 msgr2=0x7fdebc104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:34:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.496+0000 7fdec1b9f700 1 -- 172.21.15.83:0/3367465031 shutdown_connections 2024-04-25T11:34:59.043 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-25T11:34:58.496+0000 7fdec1b9f700 1 -- 172.21.15.83:0/3367465031 wait complete. 2024-04-25T11:34:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T11:34:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T11:34:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:34:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:34:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:34:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T11:34:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:34:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T11:34:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:34:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T11:34:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:34:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: ceph telemetry on 2024-04-25T11:34:59.044 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:34:59.044 INFO:teuthology.orchestra.run.smithi083.stdout:For more information see: 2024-04-25T11:34:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:34:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T11:34:59.045 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:34:59.045 INFO:teuthology.orchestra.run.smithi083.stdout:Bootstrap complete. 2024-04-25T11:34:59.076 INFO:tasks.cephadm:Fetching config... 2024-04-25T11:34:59.077 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:34:59.077 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T11:34:59.092 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T11:34:59.092 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:34:59.093 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T11:34:59.148 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T11:34:59.148 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:34:59.148 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T11:34:59.212 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T11:34:59.212 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:34:59.212 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T11:34:59.267 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T11:34:59.268 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDW0qtjjPz5Gr7fr59HtylgjXV9iF2/G/BPc/I5bYfu2pn7SaxS4Uav/cVSGfZ+jT0nwUpsUNQSqVdjrWHMubRs1J6xTDMBxSE+n4DFVoWFldsjsnnrfGUr42KLlndTr8tOY/zEkF0xhTAATa2PWL61wn5p5R/PtdcdrBB9a3fh92nhTlwz7RKwKdEDt8w0wacT/oHn3ctqJfZipN3ttHzZm8+0p6nWsgKKzrrw7w2Fj/XL6UKOl3bgodyEIp4jLf1IFSqudHBZaxPpCCiyXwK/5ulRz23R881seyYCxWTE2An4C6gK/UkF12sbXtpjec8wUwjXurq3p4TKBUNEUYl9IQd3p3+/uovr9bLEGud0OdjbwURERtt5L3rUY7v93PBRi+7cR91wh+t2JJkgM4PvBE7x8ihSl+VRHuvcbi647zXSFy4tJrGWe2Xi7Hjm85hfvuTcG0jo1UI0KFGL+aAE3q+Z9UZ0d6xrRLpsIaWzJMQx1bYqTprf13BtRqJCOM0= ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T11:34:59.339 INFO:teuthology.orchestra.run.smithi083.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDW0qtjjPz5Gr7fr59HtylgjXV9iF2/G/BPc/I5bYfu2pn7SaxS4Uav/cVSGfZ+jT0nwUpsUNQSqVdjrWHMubRs1J6xTDMBxSE+n4DFVoWFldsjsnnrfGUr42KLlndTr8tOY/zEkF0xhTAATa2PWL61wn5p5R/PtdcdrBB9a3fh92nhTlwz7RKwKdEDt8w0wacT/oHn3ctqJfZipN3ttHzZm8+0p6nWsgKKzrrw7w2Fj/XL6UKOl3bgodyEIp4jLf1IFSqudHBZaxPpCCiyXwK/5ulRz23R881seyYCxWTE2An4C6gK/UkF12sbXtpjec8wUwjXurq3p4TKBUNEUYl9IQd3p3+/uovr9bLEGud0OdjbwURERtt5L3rUY7v93PBRi+7cR91wh+t2JJkgM4PvBE7x8ihSl+VRHuvcbi647zXSFy4tJrGWe2Xi7Hjm85hfvuTcG0jo1UI0KFGL+aAE3q+Z9UZ0d6xrRLpsIaWzJMQx1bYqTprf13BtRqJCOM0= ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:34:59.352 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDW0qtjjPz5Gr7fr59HtylgjXV9iF2/G/BPc/I5bYfu2pn7SaxS4Uav/cVSGfZ+jT0nwUpsUNQSqVdjrWHMubRs1J6xTDMBxSE+n4DFVoWFldsjsnnrfGUr42KLlndTr8tOY/zEkF0xhTAATa2PWL61wn5p5R/PtdcdrBB9a3fh92nhTlwz7RKwKdEDt8w0wacT/oHn3ctqJfZipN3ttHzZm8+0p6nWsgKKzrrw7w2Fj/XL6UKOl3bgodyEIp4jLf1IFSqudHBZaxPpCCiyXwK/5ulRz23R881seyYCxWTE2An4C6gK/UkF12sbXtpjec8wUwjXurq3p4TKBUNEUYl9IQd3p3+/uovr9bLEGud0OdjbwURERtt5L3rUY7v93PBRi+7cR91wh+t2JJkgM4PvBE7x8ihSl+VRHuvcbi647zXSFy4tJrGWe2Xi7Hjm85hfvuTcG0jo1UI0KFGL+aAE3q+Z9UZ0d6xrRLpsIaWzJMQx1bYqTprf13BtRqJCOM0= ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T11:34:59.391 INFO:teuthology.orchestra.run.smithi094.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDW0qtjjPz5Gr7fr59HtylgjXV9iF2/G/BPc/I5bYfu2pn7SaxS4Uav/cVSGfZ+jT0nwUpsUNQSqVdjrWHMubRs1J6xTDMBxSE+n4DFVoWFldsjsnnrfGUr42KLlndTr8tOY/zEkF0xhTAATa2PWL61wn5p5R/PtdcdrBB9a3fh92nhTlwz7RKwKdEDt8w0wacT/oHn3ctqJfZipN3ttHzZm8+0p6nWsgKKzrrw7w2Fj/XL6UKOl3bgodyEIp4jLf1IFSqudHBZaxPpCCiyXwK/5ulRz23R881seyYCxWTE2An4C6gK/UkF12sbXtpjec8wUwjXurq3p4TKBUNEUYl9IQd3p3+/uovr9bLEGud0OdjbwURERtt5L3rUY7v93PBRi+7cR91wh+t2JJkgM4PvBE7x8ihSl+VRHuvcbi647zXSFy4tJrGWe2Xi7Hjm85hfvuTcG0jo1UI0KFGL+aAE3q+Z9UZ0d6xrRLpsIaWzJMQx1bYqTprf13BtRqJCOM0= ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:34:59.402 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDW0qtjjPz5Gr7fr59HtylgjXV9iF2/G/BPc/I5bYfu2pn7SaxS4Uav/cVSGfZ+jT0nwUpsUNQSqVdjrWHMubRs1J6xTDMBxSE+n4DFVoWFldsjsnnrfGUr42KLlndTr8tOY/zEkF0xhTAATa2PWL61wn5p5R/PtdcdrBB9a3fh92nhTlwz7RKwKdEDt8w0wacT/oHn3ctqJfZipN3ttHzZm8+0p6nWsgKKzrrw7w2Fj/XL6UKOl3bgodyEIp4jLf1IFSqudHBZaxPpCCiyXwK/5ulRz23R881seyYCxWTE2An4C6gK/UkF12sbXtpjec8wUwjXurq3p4TKBUNEUYl9IQd3p3+/uovr9bLEGud0OdjbwURERtt5L3rUY7v93PBRi+7cR91wh+t2JJkgM4PvBE7x8ihSl+VRHuvcbi647zXSFy4tJrGWe2Xi7Hjm85hfvuTcG0jo1UI0KFGL+aAE3q+Z9UZ0d6xrRLpsIaWzJMQx1bYqTprf13BtRqJCOM0= ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T11:34:59.440 INFO:teuthology.orchestra.run.smithi111.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDW0qtjjPz5Gr7fr59HtylgjXV9iF2/G/BPc/I5bYfu2pn7SaxS4Uav/cVSGfZ+jT0nwUpsUNQSqVdjrWHMubRs1J6xTDMBxSE+n4DFVoWFldsjsnnrfGUr42KLlndTr8tOY/zEkF0xhTAATa2PWL61wn5p5R/PtdcdrBB9a3fh92nhTlwz7RKwKdEDt8w0wacT/oHn3ctqJfZipN3ttHzZm8+0p6nWsgKKzrrw7w2Fj/XL6UKOl3bgodyEIp4jLf1IFSqudHBZaxPpCCiyXwK/5ulRz23R881seyYCxWTE2An4C6gK/UkF12sbXtpjec8wUwjXurq3p4TKBUNEUYl9IQd3p3+/uovr9bLEGud0OdjbwURERtt5L3rUY7v93PBRi+7cR91wh+t2JJkgM4PvBE7x8ihSl+VRHuvcbi647zXSFy4tJrGWe2Xi7Hjm85hfvuTcG0jo1UI0KFGL+aAE3q+Z9UZ0d6xrRLpsIaWzJMQx1bYqTprf13BtRqJCOM0= ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:34:59.452 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T11:34:59.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:34:59 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3367465031' entity='client.admin' 2024-04-25T11:34:59.695 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:35:02.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.101+0000 7f7c50523700 1 -- 172.21.15.83:0/3923875208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c48102f40 msgr2=0x7f7c48105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:02.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.101+0000 7f7c50523700 1 --2- 172.21.15.83:0/3923875208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c48102f40 0x7f7c48105330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009910 tx=0x7f7c44009c20 comp rx=0 tx=0).stop 2024-04-25T11:35:02.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.101+0000 7f7c50523700 1 -- 172.21.15.83:0/3923875208 shutdown_connections 2024-04-25T11:35:02.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.101+0000 7f7c50523700 1 --2- 172.21.15.83:0/3923875208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c48102f40 0x7f7c48105330 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.101+0000 7f7c50523700 1 -- 172.21.15.83:0/3923875208 >> 172.21.15.83:0/3923875208 conn(0x7f7c480fc950 msgr2=0x7f7c480fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:02.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.101+0000 7f7c50523700 1 -- 172.21.15.83:0/3923875208 shutdown_connections 2024-04-25T11:35:02.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.101+0000 7f7c50523700 1 -- 172.21.15.83:0/3923875208 wait complete. 2024-04-25T11:35:02.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.102+0000 7f7c50523700 1 Processor -- start 2024-04-25T11:35:02.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.102+0000 7f7c50523700 1 -- start start 2024-04-25T11:35:02.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.102+0000 7f7c50523700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c48102f40 0x7f7c48197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:02.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.102+0000 7f7c50523700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c48108390 con 0x7f7c48102f40 2024-04-25T11:35:02.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.102+0000 7f7c4e2bf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c48102f40 0x7f7c48197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:02.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.102+0000 7f7c4e2bf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c48102f40 0x7f7c48197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53668/0 (socket says 172.21.15.83:53668) 2024-04-25T11:35:02.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.102+0000 7f7c4e2bf700 1 -- 172.21.15.83:0/3183181690 learned_addr learned my addr 172.21.15.83:0/3183181690 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:35:02.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.102+0000 7f7c4e2bf700 1 -- 172.21.15.83:0/3183181690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c48197f70 con 0x7f7c48102f40 2024-04-25T11:35:02.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.102+0000 7f7c4e2bf700 1 --2- 172.21.15.83:0/3183181690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c48102f40 0x7f7c48197a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c44005b40 tx=0x7f7c44004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:02.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.103+0000 7f7c3f7fe700 1 -- 172.21.15.83:0/3183181690 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c44027070 con 0x7f7c48102f40 2024-04-25T11:35:02.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.103+0000 7f7c50523700 1 -- 172.21.15.83:0/3183181690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c48198260 con 0x7f7c48102f40 2024-04-25T11:35:02.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.103+0000 7f7c3f7fe700 1 -- 172.21.15.83:0/3183181690 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c4400be10 con 0x7f7c48102f40 2024-04-25T11:35:02.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.103+0000 7f7c50523700 1 -- 172.21.15.83:0/3183181690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c4819aef0 con 0x7f7c48102f40 2024-04-25T11:35:02.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.103+0000 7f7c3f7fe700 1 -- 172.21.15.83:0/3183181690 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c44022de0 con 0x7f7c48102f40 2024-04-25T11:35:02.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.103+0000 7f7c3f7fe700 1 -- 172.21.15.83:0/3183181690 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7c4402a070 con 0x7f7c48102f40 2024-04-25T11:35:02.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.104+0000 7f7c3f7fe700 1 --2- 172.21.15.83:0/3183181690 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c340396b0 0x7f7c3403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:02.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.104+0000 7f7c3f7fe700 1 -- 172.21.15.83:0/3183181690 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7c4401e030 con 0x7f7c48102f40 2024-04-25T11:35:02.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.104+0000 7f7c4dabe700 1 --2- 172.21.15.83:0/3183181690 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c340396b0 0x7f7c3403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:02.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.104+0000 7f7c50523700 1 -- 172.21.15.83:0/3183181690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c48108b10 con 0x7f7c48102f40 2024-04-25T11:35:02.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.104+0000 7f7c4dabe700 1 --2- 172.21.15.83:0/3183181690 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c340396b0 0x7f7c3403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006fd0 tx=0x7f7c38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:02.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.109+0000 7f7c3f7fe700 1 -- 172.21.15.83:0/3183181690 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c44031070 con 0x7f7c48102f40 2024-04-25T11:35:02.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.315+0000 7f7c50523700 1 -- 172.21.15.83:0/3183181690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f7c4802e090 con 0x7f7c48102f40 2024-04-25T11:35:02.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.318+0000 7f7c3f7fe700 1 -- 172.21.15.83:0/3183181690 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f7c44013070 con 0x7f7c48102f40 2024-04-25T11:35:02.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.321+0000 7f7c50523700 1 -- 172.21.15.83:0/3183181690 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c340396b0 msgr2=0x7f7c3403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:02.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.321+0000 7f7c50523700 1 --2- 172.21.15.83:0/3183181690 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c340396b0 0x7f7c3403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006fd0 tx=0x7f7c38006e40 comp rx=0 tx=0).stop 2024-04-25T11:35:02.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.321+0000 7f7c50523700 1 -- 172.21.15.83:0/3183181690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c48102f40 msgr2=0x7f7c48197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:02.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.321+0000 7f7c50523700 1 --2- 172.21.15.83:0/3183181690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c48102f40 0x7f7c48197a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7c44005b40 tx=0x7f7c44004f80 comp rx=0 tx=0).stop 2024-04-25T11:35:02.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.321+0000 7f7c50523700 1 -- 172.21.15.83:0/3183181690 shutdown_connections 2024-04-25T11:35:02.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.321+0000 7f7c50523700 1 --2- 172.21.15.83:0/3183181690 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c340396b0 0x7f7c3403bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.321+0000 7f7c50523700 1 --2- 172.21.15.83:0/3183181690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c48102f40 0x7f7c48197a30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:02.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.321+0000 7f7c50523700 1 -- 172.21.15.83:0/3183181690 >> 172.21.15.83:0/3183181690 conn(0x7f7c480fc950 msgr2=0x7f7c480fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:02.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.321+0000 7f7c50523700 1 -- 172.21.15.83:0/3183181690 shutdown_connections 2024-04-25T11:35:02.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:02.321+0000 7f7c50523700 1 -- 172.21.15.83:0/3183181690 wait complete. 2024-04-25T11:35:02.881 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T11:35:02.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T11:35:03.132 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:35:03.592 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:03 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3183181690' entity='client.admin' 2024-04-25T11:35:04.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:04.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:04.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:35:04.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:04.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:35:04.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:04.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.910+0000 7f917c90b700 1 -- 172.21.15.83:0/134397288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9178102f30 msgr2=0x7f9178105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:04.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.910+0000 7f917c90b700 1 --2- 172.21.15.83:0/134397288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9178102f30 0x7f9178105320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9168009910 tx=0x7f9168009c20 comp rx=0 tx=0).stop 2024-04-25T11:35:04.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.910+0000 7f917c90b700 1 -- 172.21.15.83:0/134397288 shutdown_connections 2024-04-25T11:35:04.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.910+0000 7f917c90b700 1 --2- 172.21.15.83:0/134397288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9178102f30 0x7f9178105320 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:04.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.910+0000 7f917c90b700 1 -- 172.21.15.83:0/134397288 >> 172.21.15.83:0/134397288 conn(0x7f91780fc940 msgr2=0x7f91780fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:04.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.910+0000 7f917c90b700 1 -- 172.21.15.83:0/134397288 shutdown_connections 2024-04-25T11:35:04.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.910+0000 7f917c90b700 1 -- 172.21.15.83:0/134397288 wait complete. 2024-04-25T11:35:04.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.911+0000 7f917c90b700 1 Processor -- start 2024-04-25T11:35:04.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.911+0000 7f917c90b700 1 -- start start 2024-04-25T11:35:04.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.911+0000 7f917c90b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9178102f30 0x7f9178197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:04.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.911+0000 7f917c90b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9178108380 con 0x7f9178102f30 2024-04-25T11:35:04.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.911+0000 7f917659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9178102f30 0x7f9178197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:04.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.911+0000 7f917659c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9178102f30 0x7f9178197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60928/0 (socket says 172.21.15.83:60928) 2024-04-25T11:35:04.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.911+0000 7f917659c700 1 -- 172.21.15.83:0/1107619052 learned_addr learned my addr 172.21.15.83:0/1107619052 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:35:04.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.911+0000 7f917659c700 1 -- 172.21.15.83:0/1107619052 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9178197fd0 con 0x7f9178102f30 2024-04-25T11:35:04.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.912+0000 7f917659c700 1 --2- 172.21.15.83:0/1107619052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9178102f30 0x7f9178197a90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9168005b40 tx=0x7f9168004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:04.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.912+0000 7f916ffff700 1 -- 172.21.15.83:0/1107619052 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9168027070 con 0x7f9178102f30 2024-04-25T11:35:04.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.912+0000 7f917c90b700 1 -- 172.21.15.83:0/1107619052 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91781982c0 con 0x7f9178102f30 2024-04-25T11:35:04.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.912+0000 7f916ffff700 1 -- 172.21.15.83:0/1107619052 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f916800be10 con 0x7f9178102f30 2024-04-25T11:35:04.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.912+0000 7f916ffff700 1 -- 172.21.15.83:0/1107619052 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9168022de0 con 0x7f9178102f30 2024-04-25T11:35:04.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.912+0000 7f917c90b700 1 -- 172.21.15.83:0/1107619052 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f917819af00 con 0x7f9178102f30 2024-04-25T11:35:04.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.913+0000 7f916ffff700 1 -- 172.21.15.83:0/1107619052 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f916802a070 con 0x7f9178102f30 2024-04-25T11:35:04.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.913+0000 7f917c90b700 1 -- 172.21.15.83:0/1107619052 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9178108b00 con 0x7f9178102f30 2024-04-25T11:35:04.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.913+0000 7f916ffff700 1 --2- 172.21.15.83:0/1107619052 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9164039650 0x7f916403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:04.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.913+0000 7f916ffff700 1 -- 172.21.15.83:0/1107619052 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f916801e030 con 0x7f9178102f30 2024-04-25T11:35:04.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.913+0000 7f916dd9b700 1 --2- 172.21.15.83:0/1107619052 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9164039650 0x7f916403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:04.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.913+0000 7f916dd9b700 1 --2- 172.21.15.83:0/1107619052 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9164039650 0x7f916403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9160006fd0 tx=0x7f9160006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:04.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:04.918+0000 7f916ffff700 1 -- 172.21.15.83:0/1107619052 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9168013070 con 0x7f9178102f30 2024-04-25T11:35:05.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:05.116+0000 7f917c90b700 1 -- 172.21.15.83:0/1107619052 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f917802e170 con 0x7f9164039650 2024-04-25T11:35:05.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:05.120+0000 7f916ffff700 1 -- 172.21.15.83:0/1107619052 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f917802e170 con 0x7f9164039650 2024-04-25T11:35:05.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:05.122+0000 7f917c90b700 1 -- 172.21.15.83:0/1107619052 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9164039650 msgr2=0x7f916403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:05.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:05.122+0000 7f917c90b700 1 --2- 172.21.15.83:0/1107619052 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9164039650 0x7f916403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9160006fd0 tx=0x7f9160006e40 comp rx=0 tx=0).stop 2024-04-25T11:35:05.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:05.122+0000 7f917c90b700 1 -- 172.21.15.83:0/1107619052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9178102f30 msgr2=0x7f9178197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:05.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:05.122+0000 7f917c90b700 1 --2- 172.21.15.83:0/1107619052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9178102f30 0x7f9178197a90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9168005b40 tx=0x7f9168004f80 comp rx=0 tx=0).stop 2024-04-25T11:35:05.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:05.122+0000 7f917c90b700 1 -- 172.21.15.83:0/1107619052 shutdown_connections 2024-04-25T11:35:05.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:05.122+0000 7f917c90b700 1 --2- 172.21.15.83:0/1107619052 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9164039650 0x7f916403bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:05.122+0000 7f917c90b700 1 --2- 172.21.15.83:0/1107619052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9178102f30 0x7f9178197a90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:05.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:05.122+0000 7f917c90b700 1 -- 172.21.15.83:0/1107619052 >> 172.21.15.83:0/1107619052 conn(0x7f91780fc940 msgr2=0x7f91780fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:05.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:05.122+0000 7f917c90b700 1 -- 172.21.15.83:0/1107619052 shutdown_connections 2024-04-25T11:35:05.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:05.122+0000 7f917c90b700 1 -- 172.21.15.83:0/1107619052 wait complete. 2024-04-25T11:35:05.854 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi094 2024-04-25T11:35:05.855 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T11:35:05.855 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/etc/ceph/ceph.conf 2024-04-25T11:35:05.874 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T11:35:05.874 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:35:05.931 INFO:tasks.cephadm:Adding host smithi094 to orchestrator... 2024-04-25T11:35:05.931 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch host add smithi094 2024-04-25T11:35:06.170 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:35:06.390 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:06 smithi083 ceph-mon[29330]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:35:06.390 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:06.390 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:35:06.390 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:06.391 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:35:06.391 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:06 smithi083 ceph-mon[29330]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-25T11:35:06.391 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:06 smithi083 ceph-mon[29330]: Updating smithi083:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:35:06.391 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:06 smithi083 ceph-mon[29330]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:35:06.391 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:06 smithi083 ceph-mon[29330]: Updating smithi083:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:35:06.391 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:06.391 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:06.392 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.680+0000 7f40dbb16700 1 -- 172.21.15.83:0/165624486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40d41033e0 msgr2=0x7f40d41037c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.680+0000 7f40dbb16700 1 --2- 172.21.15.83:0/165624486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40d41033e0 0x7f40d41037c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009910 tx=0x7f40d0009c20 comp rx=0 tx=0).stop 2024-04-25T11:35:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.680+0000 7f40dbb16700 1 -- 172.21.15.83:0/165624486 shutdown_connections 2024-04-25T11:35:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.680+0000 7f40dbb16700 1 --2- 172.21.15.83:0/165624486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40d41033e0 0x7f40d41037c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.680+0000 7f40dbb16700 1 -- 172.21.15.83:0/165624486 >> 172.21.15.83:0/165624486 conn(0x7f40d4078970 msgr2=0x7f40d4078d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.680+0000 7f40dbb16700 1 -- 172.21.15.83:0/165624486 shutdown_connections 2024-04-25T11:35:07.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.680+0000 7f40dbb16700 1 -- 172.21.15.83:0/165624486 wait complete. 2024-04-25T11:35:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.681+0000 7f40dbb16700 1 Processor -- start 2024-04-25T11:35:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.681+0000 7f40dbb16700 1 -- start start 2024-04-25T11:35:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.681+0000 7f40dbb16700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40d41033e0 0x7f40d419a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:07.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.681+0000 7f40dbb16700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f40d410ec10 con 0x7f40d41033e0 2024-04-25T11:35:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.681+0000 7f40d98b2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40d41033e0 0x7f40d419a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.681+0000 7f40d98b2700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40d41033e0 0x7f40d419a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60940/0 (socket says 172.21.15.83:60940) 2024-04-25T11:35:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.681+0000 7f40d98b2700 1 -- 172.21.15.83:0/3379677216 learned_addr learned my addr 172.21.15.83:0/3379677216 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:35:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.681+0000 7f40d98b2700 1 -- 172.21.15.83:0/3379677216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d419a6d0 con 0x7f40d41033e0 2024-04-25T11:35:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.682+0000 7f40d98b2700 1 --2- 172.21.15.83:0/3379677216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40d41033e0 0x7f40d419a190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f40d0006010 tx=0x7f40d0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.682+0000 7f40caffd700 1 -- 172.21.15.83:0/3379677216 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40d0026070 con 0x7f40d41033e0 2024-04-25T11:35:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.682+0000 7f40caffd700 1 -- 172.21.15.83:0/3379677216 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40d000bc00 con 0x7f40d41033e0 2024-04-25T11:35:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.682+0000 7f40dbb16700 1 -- 172.21.15.83:0/3379677216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d419a960 con 0x7f40d41033e0 2024-04-25T11:35:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.682+0000 7f40caffd700 1 -- 172.21.15.83:0/3379677216 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40d0003910 con 0x7f40d41033e0 2024-04-25T11:35:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.682+0000 7f40dbb16700 1 -- 172.21.15.83:0/3379677216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d419adc0 con 0x7f40d41033e0 2024-04-25T11:35:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.683+0000 7f40caffd700 1 -- 172.21.15.83:0/3379677216 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f40d0029070 con 0x7f40d41033e0 2024-04-25T11:35:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.683+0000 7f40caffd700 1 --2- 172.21.15.83:0/3379677216 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f40c0039650 0x7f40c003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.683+0000 7f40caffd700 1 -- 172.21.15.83:0/3379677216 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f40d001e070 con 0x7f40d41033e0 2024-04-25T11:35:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.683+0000 7f40dbb16700 1 -- 172.21.15.83:0/3379677216 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d4101b60 con 0x7f40d41033e0 2024-04-25T11:35:07.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.683+0000 7f40d90b1700 1 --2- 172.21.15.83:0/3379677216 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f40c0039650 0x7f40c003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:07.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.683+0000 7f40d90b1700 1 --2- 172.21.15.83:0/3379677216 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f40c0039650 0x7f40c003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f40c4006fd0 tx=0x7f40c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:07.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.688+0000 7f40caffd700 1 -- 172.21.15.83:0/3379677216 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40d0030070 con 0x7f40d41033e0 2024-04-25T11:35:07.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:07.890+0000 7f40dbb16700 1 -- 172.21.15.83:0/3379677216 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi094", "target": ["mon-mgr", ""]}) -- 0x7f40d419b4f0 con 0x7f40c0039650 2024-04-25T11:35:08.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:08 smithi083 ceph-mon[29330]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi094", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:35:09.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:09 smithi083 ceph-mon[29330]: Deploying cephadm binary to smithi094 2024-04-25T11:35:11.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:10 smithi083 ceph-mon[29330]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:12.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:12.028+0000 7f40caffd700 1 -- 172.21.15.83:0/3379677216 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f40d419b4f0 con 0x7f40c0039650 2024-04-25T11:35:12.017 INFO:teuthology.orchestra.run.smithi083.stdout:Added host 'smithi094' with addr '172.21.15.94' 2024-04-25T11:35:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:12.030+0000 7f40dbb16700 1 -- 172.21.15.83:0/3379677216 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f40c0039650 msgr2=0x7f40c003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:12.030+0000 7f40dbb16700 1 --2- 172.21.15.83:0/3379677216 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f40c0039650 0x7f40c003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f40c4006fd0 tx=0x7f40c4006e40 comp rx=0 tx=0).stop 2024-04-25T11:35:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:12.030+0000 7f40dbb16700 1 -- 172.21.15.83:0/3379677216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40d41033e0 msgr2=0x7f40d419a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:12.030+0000 7f40dbb16700 1 --2- 172.21.15.83:0/3379677216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40d41033e0 0x7f40d419a190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f40d0006010 tx=0x7f40d0005780 comp rx=0 tx=0).stop 2024-04-25T11:35:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:12.030+0000 7f40dbb16700 1 -- 172.21.15.83:0/3379677216 shutdown_connections 2024-04-25T11:35:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:12.030+0000 7f40dbb16700 1 --2- 172.21.15.83:0/3379677216 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f40c0039650 0x7f40c003bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:12.030+0000 7f40dbb16700 1 --2- 172.21.15.83:0/3379677216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40d41033e0 0x7f40d419a190 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:12.030+0000 7f40dbb16700 1 -- 172.21.15.83:0/3379677216 >> 172.21.15.83:0/3379677216 conn(0x7f40d4078970 msgr2=0x7f40d410a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:12.030+0000 7f40dbb16700 1 -- 172.21.15.83:0/3379677216 shutdown_connections 2024-04-25T11:35:12.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:12.030+0000 7f40dbb16700 1 -- 172.21.15.83:0/3379677216 wait complete. 2024-04-25T11:35:12.609 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T11:35:12.845 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:35:13.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:12 smithi083 ceph-mon[29330]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:13.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:13.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:35:13.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:13.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:13 smithi083 ceph-mon[29330]: Added host smithi094 2024-04-25T11:35:13.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:13 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:14.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.366+0000 7fa9ed21a700 1 -- 172.21.15.83:0/4025245717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81015b0 msgr2=0x7fa9e8101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:14.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.366+0000 7fa9ed21a700 1 --2- 172.21.15.83:0/4025245717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81015b0 0x7fa9e8101990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009910 tx=0x7fa9d8009c20 comp rx=0 tx=0).stop 2024-04-25T11:35:14.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.366+0000 7fa9ed21a700 1 -- 172.21.15.83:0/4025245717 shutdown_connections 2024-04-25T11:35:14.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.366+0000 7fa9ed21a700 1 --2- 172.21.15.83:0/4025245717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81015b0 0x7fa9e8101990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.366+0000 7fa9ed21a700 1 -- 172.21.15.83:0/4025245717 >> 172.21.15.83:0/4025245717 conn(0x7fa9e80fce20 msgr2=0x7fa9e80ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:14.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.366+0000 7fa9ed21a700 1 -- 172.21.15.83:0/4025245717 shutdown_connections 2024-04-25T11:35:14.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.366+0000 7fa9ed21a700 1 -- 172.21.15.83:0/4025245717 wait complete. 2024-04-25T11:35:14.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.367+0000 7fa9ed21a700 1 Processor -- start 2024-04-25T11:35:14.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.367+0000 7fa9ed21a700 1 -- start start 2024-04-25T11:35:14.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.367+0000 7fa9ed21a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81015b0 0x7fa9e8198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:14.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.367+0000 7fa9ed21a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e810b830 con 0x7fa9e81015b0 2024-04-25T11:35:14.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.367+0000 7fa9e6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81015b0 0x7fa9e8198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:14.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.367+0000 7fa9e6d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81015b0 0x7fa9e8198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50078/0 (socket says 172.21.15.83:50078) 2024-04-25T11:35:14.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.367+0000 7fa9e6d9d700 1 -- 172.21.15.83:0/1805386652 learned_addr learned my addr 172.21.15.83:0/1805386652 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:35:14.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.368+0000 7fa9e6d9d700 1 -- 172.21.15.83:0/1805386652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e81991d0 con 0x7fa9e81015b0 2024-04-25T11:35:14.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.368+0000 7fa9e6d9d700 1 --2- 172.21.15.83:0/1805386652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81015b0 0x7fa9e8198c90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80056d0 tx=0x7fa9d80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:14.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.368+0000 7fa9cffff700 1 -- 172.21.15.83:0/1805386652 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9d8026070 con 0x7fa9e81015b0 2024-04-25T11:35:14.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.368+0000 7fa9cffff700 1 -- 172.21.15.83:0/1805386652 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9d800bba0 con 0x7fa9e81015b0 2024-04-25T11:35:14.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.368+0000 7fa9cffff700 1 -- 172.21.15.83:0/1805386652 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9d8003920 con 0x7fa9e81015b0 2024-04-25T11:35:14.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.368+0000 7fa9ed21a700 1 -- 172.21.15.83:0/1805386652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e81994c0 con 0x7fa9e81015b0 2024-04-25T11:35:14.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.368+0000 7fa9ed21a700 1 -- 172.21.15.83:0/1805386652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e8199980 con 0x7fa9e81015b0 2024-04-25T11:35:14.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.369+0000 7fa9cffff700 1 -- 172.21.15.83:0/1805386652 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa9d8029070 con 0x7fa9e81015b0 2024-04-25T11:35:14.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.369+0000 7fa9ed21a700 1 -- 172.21.15.83:0/1805386652 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e8102a90 con 0x7fa9e81015b0 2024-04-25T11:35:14.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.369+0000 7fa9cffff700 1 --2- 172.21.15.83:0/1805386652 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa9d0039650 0x7fa9d003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:14.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.369+0000 7fa9cffff700 1 -- 172.21.15.83:0/1805386652 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa9d801e070 con 0x7fa9e81015b0 2024-04-25T11:35:14.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.369+0000 7fa9e659c700 1 --2- 172.21.15.83:0/1805386652 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa9d0039650 0x7fa9d003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:14.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.370+0000 7fa9e659c700 1 --2- 172.21.15.83:0/1805386652 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa9d0039650 0x7fa9d003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc006fd0 tx=0x7fa9dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:14.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.374+0000 7fa9cffff700 1 -- 172.21.15.83:0/1805386652 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9d8013070 con 0x7fa9e81015b0 2024-04-25T11:35:14.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.577+0000 7fa9ed21a700 1 -- 172.21.15.83:0/1805386652 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9e806af60 con 0x7fa9d0039650 2024-04-25T11:35:14.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.578+0000 7fa9cffff700 1 -- 172.21.15.83:0/1805386652 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fa9e806af60 con 0x7fa9d0039650 2024-04-25T11:35:14.568 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:35:14.568 INFO:teuthology.orchestra.run.smithi083.stdout:[{"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}] 2024-04-25T11:35:14.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.580+0000 7fa9ed21a700 1 -- 172.21.15.83:0/1805386652 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa9d0039650 msgr2=0x7fa9d003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.580+0000 7fa9ed21a700 1 --2- 172.21.15.83:0/1805386652 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa9d0039650 0x7fa9d003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc006fd0 tx=0x7fa9dc006e40 comp rx=0 tx=0).stop 2024-04-25T11:35:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.580+0000 7fa9ed21a700 1 -- 172.21.15.83:0/1805386652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81015b0 msgr2=0x7fa9e8198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.580+0000 7fa9ed21a700 1 --2- 172.21.15.83:0/1805386652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81015b0 0x7fa9e8198c90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80056d0 tx=0x7fa9d80057b0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.580+0000 7fa9ed21a700 1 -- 172.21.15.83:0/1805386652 shutdown_connections 2024-04-25T11:35:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.580+0000 7fa9ed21a700 1 --2- 172.21.15.83:0/1805386652 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa9d0039650 0x7fa9d003bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.580+0000 7fa9ed21a700 1 --2- 172.21.15.83:0/1805386652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9e81015b0 0x7fa9e8198c90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.580+0000 7fa9ed21a700 1 -- 172.21.15.83:0/1805386652 >> 172.21.15.83:0/1805386652 conn(0x7fa9e80fce20 msgr2=0x7fa9e8102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.580+0000 7fa9ed21a700 1 -- 172.21.15.83:0/1805386652 shutdown_connections 2024-04-25T11:35:14.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:14.580+0000 7fa9ed21a700 1 -- 172.21.15.83:0/1805386652 wait complete. 2024-04-25T11:35:15.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:14 smithi083 ceph-mon[29330]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:15.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:14 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:15.219 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi111 2024-04-25T11:35:15.219 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-25T11:35:15.219 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/etc/ceph/ceph.conf 2024-04-25T11:35:15.238 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-25T11:35:15.239 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:35:15.297 INFO:tasks.cephadm:Adding host smithi111 to orchestrator... 2024-04-25T11:35:15.297 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch host add smithi111 2024-04-25T11:35:15.529 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:35:15.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:15 smithi083 ceph-mon[29330]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:35:16.843 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:16 smithi083 ceph-mon[29330]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:17.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.054+0000 7f5f624bc700 1 -- 172.21.15.83:0/2884074542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f5c1084e0 msgr2=0x7f5f5c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:17.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.054+0000 7f5f624bc700 1 --2- 172.21.15.83:0/2884074542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f5c1084e0 0x7f5f5c1088c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009910 tx=0x7f5f50009c20 comp rx=0 tx=0).stop 2024-04-25T11:35:17.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.054+0000 7f5f624bc700 1 -- 172.21.15.83:0/2884074542 shutdown_connections 2024-04-25T11:35:17.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.054+0000 7f5f624bc700 1 --2- 172.21.15.83:0/2884074542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f5c1084e0 0x7f5f5c1088c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:17.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.054+0000 7f5f624bc700 1 -- 172.21.15.83:0/2884074542 >> 172.21.15.83:0/2884074542 conn(0x7f5f5c0fdf80 msgr2=0x7f5f5c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:17.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.054+0000 7f5f624bc700 1 -- 172.21.15.83:0/2884074542 shutdown_connections 2024-04-25T11:35:17.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.054+0000 7f5f624bc700 1 -- 172.21.15.83:0/2884074542 wait complete. 2024-04-25T11:35:17.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.054+0000 7f5f624bc700 1 Processor -- start 2024-04-25T11:35:17.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.055+0000 7f5f624bc700 1 -- start start 2024-04-25T11:35:17.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.055+0000 7f5f624bc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f5c1084e0 0x7f5f5c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:17.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.055+0000 7f5f624bc700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c10c940 con 0x7f5f5c1084e0 2024-04-25T11:35:17.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.055+0000 7f5f5bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f5c1084e0 0x7f5f5c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:17.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.055+0000 7f5f5bfff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f5c1084e0 0x7f5f5c197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50096/0 (socket says 172.21.15.83:50096) 2024-04-25T11:35:17.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.055+0000 7f5f5bfff700 1 -- 172.21.15.83:0/3808274245 learned_addr learned my addr 172.21.15.83:0/3808274245 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:35:17.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.055+0000 7f5f5bfff700 1 -- 172.21.15.83:0/3808274245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5c1983f0 con 0x7f5f5c1084e0 2024-04-25T11:35:17.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.055+0000 7f5f5bfff700 1 --2- 172.21.15.83:0/3808274245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f5c1084e0 0x7f5f5c197eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5f500056d0 tx=0x7f5f500057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:17.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.056+0000 7f5f597fa700 1 -- 172.21.15.83:0/3808274245 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f50026070 con 0x7f5f5c1084e0 2024-04-25T11:35:17.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.056+0000 7f5f597fa700 1 -- 172.21.15.83:0/3808274245 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f5000bba0 con 0x7f5f5c1084e0 2024-04-25T11:35:17.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.056+0000 7f5f624bc700 1 -- 172.21.15.83:0/3808274245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5c1986e0 con 0x7f5f5c1084e0 2024-04-25T11:35:17.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.056+0000 7f5f597fa700 1 -- 172.21.15.83:0/3808274245 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f50003920 con 0x7f5f5c1084e0 2024-04-25T11:35:17.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.056+0000 7f5f624bc700 1 -- 172.21.15.83:0/3808274245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f5c19c780 con 0x7f5f5c1084e0 2024-04-25T11:35:17.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.056+0000 7f5f597fa700 1 -- 172.21.15.83:0/3808274245 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5f50029070 con 0x7f5f5c1084e0 2024-04-25T11:35:17.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.057+0000 7f5f597fa700 1 --2- 172.21.15.83:0/3808274245 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5f44039650 0x7f5f4403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:17.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.057+0000 7f5f597fa700 1 -- 172.21.15.83:0/3808274245 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5f5001e070 con 0x7f5f5c1084e0 2024-04-25T11:35:17.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.057+0000 7f5f5b7fe700 1 --2- 172.21.15.83:0/3808274245 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5f44039650 0x7f5f4403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:17.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.057+0000 7f5f624bc700 1 -- 172.21.15.83:0/3808274245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f5c103b90 con 0x7f5f5c1084e0 2024-04-25T11:35:17.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.057+0000 7f5f5b7fe700 1 --2- 172.21.15.83:0/3808274245 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5f44039650 0x7f5f4403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c006fd0 tx=0x7f5f4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:17.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.062+0000 7f5f597fa700 1 -- 172.21.15.83:0/3808274245 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f50030070 con 0x7f5f5c1084e0 2024-04-25T11:35:17.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:17.266+0000 7f5f624bc700 1 -- 172.21.15.83:0/3808274245 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi111", "target": ["mon-mgr", ""]}) -- 0x7f5f5c199210 con 0x7f5f44039650 2024-04-25T11:35:19.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:18 smithi083 ceph-mon[29330]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi111", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:35:19.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:18 smithi083 ceph-mon[29330]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:19.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:19.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:19.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:19.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:19.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:35:19.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:19.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:35:19.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:18 smithi083 ceph-mon[29330]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-25T11:35:19.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:18 smithi083 ceph-mon[29330]: Deploying cephadm binary to smithi111 2024-04-25T11:35:20.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:19 smithi083 ceph-mon[29330]: Updating smithi094:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:35:20.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:19 smithi083 ceph-mon[29330]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:35:21.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:20 smithi083 ceph-mon[29330]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:21.467 INFO:teuthology.orchestra.run.smithi083.stdout:Added host 'smithi111' with addr '172.21.15.111' 2024-04-25T11:35:21.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:21.478+0000 7f5f597fa700 1 -- 172.21.15.83:0/3808274245 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f5f5c199210 con 0x7f5f44039650 2024-04-25T11:35:21.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:21.481+0000 7f5f624bc700 1 -- 172.21.15.83:0/3808274245 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5f44039650 msgr2=0x7f5f4403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:21.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:21.481+0000 7f5f624bc700 1 --2- 172.21.15.83:0/3808274245 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5f44039650 0x7f5f4403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c006fd0 tx=0x7f5f4c006e40 comp rx=0 tx=0).stop 2024-04-25T11:35:21.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:21.481+0000 7f5f624bc700 1 -- 172.21.15.83:0/3808274245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f5c1084e0 msgr2=0x7f5f5c197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:21.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:21.481+0000 7f5f624bc700 1 --2- 172.21.15.83:0/3808274245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f5c1084e0 0x7f5f5c197eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5f500056d0 tx=0x7f5f500057b0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:21.481+0000 7f5f624bc700 1 -- 172.21.15.83:0/3808274245 shutdown_connections 2024-04-25T11:35:21.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:21.481+0000 7f5f624bc700 1 --2- 172.21.15.83:0/3808274245 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5f44039650 0x7f5f4403bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:21.481+0000 7f5f624bc700 1 --2- 172.21.15.83:0/3808274245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f5c1084e0 0x7f5f5c197eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:21.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:21.481+0000 7f5f624bc700 1 -- 172.21.15.83:0/3808274245 >> 172.21.15.83:0/3808274245 conn(0x7f5f5c0fdf80 msgr2=0x7f5f5c10a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:21.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:21.481+0000 7f5f624bc700 1 -- 172.21.15.83:0/3808274245 shutdown_connections 2024-04-25T11:35:21.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:21.481+0000 7f5f624bc700 1 -- 172.21.15.83:0/3808274245 wait complete. 2024-04-25T11:35:22.144 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:21 smithi083 ceph-mon[29330]: Updating smithi094:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:35:22.145 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:21 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:22.145 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:21 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:22.145 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:21 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:22.145 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:21 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:22.145 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:21 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:35:22.172 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T11:35:22.404 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:35:23.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:22 smithi083 ceph-mon[29330]: Added host smithi111 2024-04-25T11:35:23.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:22 smithi083 ceph-mon[29330]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:23.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:22 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:23.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:22 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.917+0000 7efdab008700 1 -- 172.21.15.83:0/4177153900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda41015f0 msgr2=0x7efda41019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.917+0000 7efdab008700 1 --2- 172.21.15.83:0/4177153900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda41015f0 0x7efda41019d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efd98009910 tx=0x7efd98009c20 comp rx=0 tx=0).stop 2024-04-25T11:35:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.917+0000 7efdab008700 1 -- 172.21.15.83:0/4177153900 shutdown_connections 2024-04-25T11:35:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.917+0000 7efdab008700 1 --2- 172.21.15.83:0/4177153900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda41015f0 0x7efda41019d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.917+0000 7efdab008700 1 -- 172.21.15.83:0/4177153900 >> 172.21.15.83:0/4177153900 conn(0x7efda4076ea0 msgr2=0x7efda40772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.917+0000 7efdab008700 1 -- 172.21.15.83:0/4177153900 shutdown_connections 2024-04-25T11:35:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.917+0000 7efdab008700 1 -- 172.21.15.83:0/4177153900 wait complete. 2024-04-25T11:35:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.917+0000 7efdab008700 1 Processor -- start 2024-04-25T11:35:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.917+0000 7efdab008700 1 -- start start 2024-04-25T11:35:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.918+0000 7efdab008700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda41015f0 0x7efda4197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.918+0000 7efdab008700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efda410b820 con 0x7efda41015f0 2024-04-25T11:35:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.918+0000 7efda8da4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda41015f0 0x7efda4197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.918+0000 7efda8da4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda41015f0 0x7efda4197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35638/0 (socket says 172.21.15.83:35638) 2024-04-25T11:35:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.918+0000 7efda8da4700 1 -- 172.21.15.83:0/2482665189 learned_addr learned my addr 172.21.15.83:0/2482665189 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:35:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.918+0000 7efda8da4700 1 -- 172.21.15.83:0/2482665189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda4198400 con 0x7efda41015f0 2024-04-25T11:35:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.918+0000 7efda8da4700 1 --2- 172.21.15.83:0/2482665189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda41015f0 0x7efda4197ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efd98006010 tx=0x7efd98005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.918+0000 7efda1ffb700 1 -- 172.21.15.83:0/2482665189 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd98026070 con 0x7efda41015f0 2024-04-25T11:35:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.918+0000 7efdab008700 1 -- 172.21.15.83:0/2482665189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda41986f0 con 0x7efda41015f0 2024-04-25T11:35:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.918+0000 7efda1ffb700 1 -- 172.21.15.83:0/2482665189 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9800bba0 con 0x7efda41015f0 2024-04-25T11:35:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.918+0000 7efda1ffb700 1 -- 172.21.15.83:0/2482665189 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd98003910 con 0x7efda41015f0 2024-04-25T11:35:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.919+0000 7efdab008700 1 -- 172.21.15.83:0/2482665189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda419c790 con 0x7efda41015f0 2024-04-25T11:35:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.919+0000 7efda1ffb700 1 -- 172.21.15.83:0/2482665189 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7efd98029070 con 0x7efda41015f0 2024-04-25T11:35:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.919+0000 7efdab008700 1 -- 172.21.15.83:0/2482665189 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda4102a80 con 0x7efda41015f0 2024-04-25T11:35:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.920+0000 7efda1ffb700 1 --2- 172.21.15.83:0/2482665189 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efd94039650 0x7efd9403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.920+0000 7efda1ffb700 1 -- 172.21.15.83:0/2482665189 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efd9801e070 con 0x7efda41015f0 2024-04-25T11:35:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.920+0000 7efda3fff700 1 --2- 172.21.15.83:0/2482665189 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efd94039650 0x7efd9403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.920+0000 7efda3fff700 1 --2- 172.21.15.83:0/2482665189 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efd94039650 0x7efd9403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efd90006fd0 tx=0x7efd90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:23.925+0000 7efda1ffb700 1 -- 172.21.15.83:0/2482665189 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd98013070 con 0x7efda41015f0 2024-04-25T11:35:24.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:24.132+0000 7efdab008700 1 -- 172.21.15.83:0/2482665189 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efda4199220 con 0x7efd94039650 2024-04-25T11:35:24.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:24.133+0000 7efda1ffb700 1 -- 172.21.15.83:0/2482665189 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7efda4199220 con 0x7efd94039650 2024-04-25T11:35:24.121 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:35:24.121 INFO:teuthology.orchestra.run.smithi083.stdout:[{"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}] 2024-04-25T11:35:24.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:24.135+0000 7efdab008700 1 -- 172.21.15.83:0/2482665189 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efd94039650 msgr2=0x7efd9403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:24.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:24.135+0000 7efdab008700 1 --2- 172.21.15.83:0/2482665189 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efd94039650 0x7efd9403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efd90006fd0 tx=0x7efd90006e40 comp rx=0 tx=0).stop 2024-04-25T11:35:24.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:24.135+0000 7efdab008700 1 -- 172.21.15.83:0/2482665189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda41015f0 msgr2=0x7efda4197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:24.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:24.135+0000 7efdab008700 1 --2- 172.21.15.83:0/2482665189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda41015f0 0x7efda4197ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efd98006010 tx=0x7efd98005780 comp rx=0 tx=0).stop 2024-04-25T11:35:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:24.135+0000 7efdab008700 1 -- 172.21.15.83:0/2482665189 shutdown_connections 2024-04-25T11:35:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:24.135+0000 7efdab008700 1 --2- 172.21.15.83:0/2482665189 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efd94039650 0x7efd9403bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:24.135+0000 7efdab008700 1 --2- 172.21.15.83:0/2482665189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efda41015f0 0x7efda4197ec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:24.135+0000 7efdab008700 1 -- 172.21.15.83:0/2482665189 >> 172.21.15.83:0/2482665189 conn(0x7efda4076ea0 msgr2=0x7efda4069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:24.135+0000 7efdab008700 1 -- 172.21.15.83:0/2482665189 shutdown_connections 2024-04-25T11:35:24.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:24.135+0000 7efdab008700 1 -- 172.21.15.83:0/2482665189 wait complete. 2024-04-25T11:35:24.710 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T11:35:24.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T11:35:24.942 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:35:25.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:24 smithi083 ceph-mon[29330]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:25.042 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:24 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:26.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:25 smithi083 ceph-mon[29330]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:35:26.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.464+0000 7feb8ee34700 1 -- 172.21.15.83:0/1324599307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb881084e0 msgr2=0x7feb881088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.464+0000 7feb8ee34700 1 --2- 172.21.15.83:0/1324599307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb881084e0 0x7feb881088c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009910 tx=0x7feb7c009c20 comp rx=0 tx=0).stop 2024-04-25T11:35:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.464+0000 7feb8ee34700 1 -- 172.21.15.83:0/1324599307 shutdown_connections 2024-04-25T11:35:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.464+0000 7feb8ee34700 1 --2- 172.21.15.83:0/1324599307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb881084e0 0x7feb881088c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.464+0000 7feb8ee34700 1 -- 172.21.15.83:0/1324599307 >> 172.21.15.83:0/1324599307 conn(0x7feb880fdf80 msgr2=0x7feb881003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.464+0000 7feb8ee34700 1 -- 172.21.15.83:0/1324599307 shutdown_connections 2024-04-25T11:35:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.464+0000 7feb8ee34700 1 -- 172.21.15.83:0/1324599307 wait complete. 2024-04-25T11:35:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.465+0000 7feb8ee34700 1 Processor -- start 2024-04-25T11:35:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.465+0000 7feb8ee34700 1 -- start start 2024-04-25T11:35:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.465+0000 7feb8ee34700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb881084e0 0x7feb88197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.465+0000 7feb8ee34700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb8810c940 con 0x7feb881084e0 2024-04-25T11:35:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.465+0000 7feb8cbd0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb881084e0 0x7feb88197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:26.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.465+0000 7feb8cbd0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb881084e0 0x7feb88197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35644/0 (socket says 172.21.15.83:35644) 2024-04-25T11:35:26.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.465+0000 7feb8cbd0700 1 -- 172.21.15.83:0/2449830992 learned_addr learned my addr 172.21.15.83:0/2449830992 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:35:26.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.466+0000 7feb8cbd0700 1 -- 172.21.15.83:0/2449830992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb881983f0 con 0x7feb881084e0 2024-04-25T11:35:26.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.466+0000 7feb8cbd0700 1 --2- 172.21.15.83:0/2449830992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb881084e0 0x7feb88197eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0056d0 tx=0x7feb7c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.466+0000 7feb85ffb700 1 -- 172.21.15.83:0/2449830992 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb7c026070 con 0x7feb881084e0 2024-04-25T11:35:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.466+0000 7feb85ffb700 1 -- 172.21.15.83:0/2449830992 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb7c00bba0 con 0x7feb881084e0 2024-04-25T11:35:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.466+0000 7feb85ffb700 1 -- 172.21.15.83:0/2449830992 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb7c003920 con 0x7feb881084e0 2024-04-25T11:35:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.466+0000 7feb8ee34700 1 -- 172.21.15.83:0/2449830992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb881986e0 con 0x7feb881084e0 2024-04-25T11:35:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.466+0000 7feb8ee34700 1 -- 172.21.15.83:0/2449830992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb8819c780 con 0x7feb881084e0 2024-04-25T11:35:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.467+0000 7feb85ffb700 1 -- 172.21.15.83:0/2449830992 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7feb7c029070 con 0x7feb881084e0 2024-04-25T11:35:26.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.467+0000 7feb8ee34700 1 -- 172.21.15.83:0/2449830992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb88103b90 con 0x7feb881084e0 2024-04-25T11:35:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.467+0000 7feb85ffb700 1 --2- 172.21.15.83:0/2449830992 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7feb780396b0 0x7feb7803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.467+0000 7feb85ffb700 1 -- 172.21.15.83:0/2449830992 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feb7c01e070 con 0x7feb881084e0 2024-04-25T11:35:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.467+0000 7feb87fff700 1 --2- 172.21.15.83:0/2449830992 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7feb780396b0 0x7feb7803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:26.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.468+0000 7feb87fff700 1 --2- 172.21.15.83:0/2449830992 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7feb780396b0 0x7feb7803bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feb74006fd0 tx=0x7feb74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:26.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.472+0000 7feb85ffb700 1 -- 172.21.15.83:0/2449830992 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb7c013070 con 0x7feb881084e0 2024-04-25T11:35:26.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.674+0000 7feb8ee34700 1 -- 172.21.15.83:0/2449830992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7feb88199210 con 0x7feb881084e0 2024-04-25T11:35:26.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.907+0000 7feb85ffb700 1 -- 172.21.15.83:0/2449830992 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7feb7c0282c0 con 0x7feb881084e0 2024-04-25T11:35:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.910+0000 7feb8ee34700 1 -- 172.21.15.83:0/2449830992 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7feb780396b0 msgr2=0x7feb7803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.910+0000 7feb8ee34700 1 --2- 172.21.15.83:0/2449830992 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7feb780396b0 0x7feb7803bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feb74006fd0 tx=0x7feb74006e40 comp rx=0 tx=0).stop 2024-04-25T11:35:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.910+0000 7feb8ee34700 1 -- 172.21.15.83:0/2449830992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb881084e0 msgr2=0x7feb88197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:26.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.910+0000 7feb8ee34700 1 --2- 172.21.15.83:0/2449830992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb881084e0 0x7feb88197eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0056d0 tx=0x7feb7c0057b0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.910+0000 7feb8ee34700 1 -- 172.21.15.83:0/2449830992 shutdown_connections 2024-04-25T11:35:26.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.910+0000 7feb8ee34700 1 --2- 172.21.15.83:0/2449830992 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7feb780396b0 0x7feb7803bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.910+0000 7feb8ee34700 1 --2- 172.21.15.83:0/2449830992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb881084e0 0x7feb88197eb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:26.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.910+0000 7feb8ee34700 1 -- 172.21.15.83:0/2449830992 >> 172.21.15.83:0/2449830992 conn(0x7feb880fdf80 msgr2=0x7feb8810a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:26.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.910+0000 7feb8ee34700 1 -- 172.21.15.83:0/2449830992 shutdown_connections 2024-04-25T11:35:26.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:26.910+0000 7feb8ee34700 1 -- 172.21.15.83:0/2449830992 wait complete. 2024-04-25T11:35:26.899 INFO:teuthology.orchestra.run.smithi083.stderr:adjusted tunables profile to default 2024-04-25T11:35:27.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:26 smithi083 ceph-mon[29330]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:27.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:26 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2449830992' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T11:35:27.480 INFO:tasks.cephadm:Adding mon.a on smithi083 2024-04-25T11:35:27.481 INFO:tasks.cephadm:Adding mon.b on smithi094 2024-04-25T11:35:27.481 INFO:tasks.cephadm:Adding mon.c on smithi111 2024-04-25T11:35:27.481 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi083:172.21.15.83=a;smithi094:172.21.15.94=b;smithi111:172.21.15.111=c' 2024-04-25T11:35:27.710 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:35:28.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:27 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2449830992' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T11:35:28.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:27 smithi083 ceph-mon[29330]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:35:28.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:27 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:28.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:27 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:28.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:27 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:28.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:27 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:28.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:27 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi111", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:35:28.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:27 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:28.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:27 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:35:29.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.221+0000 7f257d1d0700 1 -- 172.21.15.111:0/2432409357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2578108550 msgr2=0x7f2578108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.221+0000 7f257d1d0700 1 --2- 172.21.15.111:0/2432409357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2578108550 0x7f2578108930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2568009910 tx=0x7f2568009c20 comp rx=0 tx=0).stop 2024-04-25T11:35:29.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.221+0000 7f257d1d0700 1 -- 172.21.15.111:0/2432409357 shutdown_connections 2024-04-25T11:35:29.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.221+0000 7f257d1d0700 1 --2- 172.21.15.111:0/2432409357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2578108550 0x7f2578108930 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.221+0000 7f257d1d0700 1 -- 172.21.15.111:0/2432409357 >> 172.21.15.111:0/2432409357 conn(0x7f25780fdf70 msgr2=0x7f2578100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:29.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.221+0000 7f257d1d0700 1 -- 172.21.15.111:0/2432409357 shutdown_connections 2024-04-25T11:35:29.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.221+0000 7f257d1d0700 1 -- 172.21.15.111:0/2432409357 wait complete. 2024-04-25T11:35:29.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.222+0000 7f257d1d0700 1 Processor -- start 2024-04-25T11:35:29.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.222+0000 7f257d1d0700 1 -- start start 2024-04-25T11:35:29.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.222+0000 7f257d1d0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2578108550 0x7f2578197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.222+0000 7f257d1d0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f257810c9c0 con 0x7f2578108550 2024-04-25T11:35:29.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.223+0000 7f2576d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2578108550 0x7f2578197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.223+0000 7f2576d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2578108550 0x7f2578197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.111:49960/0 (socket says 172.21.15.111:49960) 2024-04-25T11:35:29.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.223+0000 7f2576d9d700 1 -- 172.21.15.111:0/1324914403 learned_addr learned my addr 172.21.15.111:0/1324914403 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-25T11:35:29.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.223+0000 7f2576d9d700 1 -- 172.21.15.111:0/1324914403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25781983a0 con 0x7f2578108550 2024-04-25T11:35:29.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.223+0000 7f2576d9d700 1 --2- 172.21.15.111:0/1324914403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2578108550 0x7f2578197e60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2568006010 tx=0x7f2568005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.223+0000 7f255ffff700 1 -- 172.21.15.111:0/1324914403 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2568026070 con 0x7f2578108550 2024-04-25T11:35:29.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.223+0000 7f257d1d0700 1 -- 172.21.15.111:0/1324914403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2578198630 con 0x7f2578108550 2024-04-25T11:35:29.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.223+0000 7f255ffff700 1 -- 172.21.15.111:0/1324914403 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f256800bc00 con 0x7f2578108550 2024-04-25T11:35:29.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.223+0000 7f255ffff700 1 -- 172.21.15.111:0/1324914403 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2568003910 con 0x7f2578108550 2024-04-25T11:35:29.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.223+0000 7f257d1d0700 1 -- 172.21.15.111:0/1324914403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f257819c730 con 0x7f2578108550 2024-04-25T11:35:29.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.224+0000 7f257d1d0700 1 -- 172.21.15.111:0/1324914403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2578103bc0 con 0x7f2578108550 2024-04-25T11:35:29.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.224+0000 7f255ffff700 1 -- 172.21.15.111:0/1324914403 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2568029070 con 0x7f2578108550 2024-04-25T11:35:29.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.224+0000 7f255ffff700 1 --2- 172.21.15.111:0/1324914403 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f2560039650 0x7f256003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:29.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.224+0000 7f255ffff700 1 -- 172.21.15.111:0/1324914403 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f256801e070 con 0x7f2578108550 2024-04-25T11:35:29.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.225+0000 7f257659c700 1 --2- 172.21.15.111:0/1324914403 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f2560039650 0x7f256003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:29.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.225+0000 7f257659c700 1 --2- 172.21.15.111:0/1324914403 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f2560039650 0x7f256003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f256c006fd0 tx=0x7f256c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:29.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.230+0000 7f255ffff700 1 -- 172.21.15.111:0/1324914403 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2568013070 con 0x7f2578108550 2024-04-25T11:35:29.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:28 smithi083 ceph-mon[29330]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-25T11:35:29.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:28 smithi083 ceph-mon[29330]: Updating smithi111:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:35:29.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:28 smithi083 ceph-mon[29330]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:35:29.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:28 smithi083 ceph-mon[29330]: Updating smithi111:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:35:29.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:28 smithi083 ceph-mon[29330]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:29.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:28 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:29.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:28 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:29.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:28 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:29.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.439+0000 7f257d1d0700 1 -- 172.21.15.111:0/1324914403 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi094:172.21.15.94=b;smithi111:172.21.15.111=c", "target": ["mon-mgr", ""]}) -- 0x7f25781991c0 con 0x7f2560039650 2024-04-25T11:35:29.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.447+0000 7f255ffff700 1 -- 172.21.15.111:0/1324914403 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f25781991c0 con 0x7f2560039650 2024-04-25T11:35:29.434 INFO:teuthology.orchestra.run.smithi111.stdout:Scheduled mon update... 2024-04-25T11:35:29.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.449+0000 7f257d1d0700 1 -- 172.21.15.111:0/1324914403 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f2560039650 msgr2=0x7f256003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.449+0000 7f257d1d0700 1 --2- 172.21.15.111:0/1324914403 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f2560039650 0x7f256003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f256c006fd0 tx=0x7f256c006e40 comp rx=0 tx=0).stop 2024-04-25T11:35:29.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.449+0000 7f257d1d0700 1 -- 172.21.15.111:0/1324914403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2578108550 msgr2=0x7f2578197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:29.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.449+0000 7f257d1d0700 1 --2- 172.21.15.111:0/1324914403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2578108550 0x7f2578197e60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2568006010 tx=0x7f2568005780 comp rx=0 tx=0).stop 2024-04-25T11:35:29.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.449+0000 7f257d1d0700 1 -- 172.21.15.111:0/1324914403 shutdown_connections 2024-04-25T11:35:29.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.449+0000 7f257d1d0700 1 --2- 172.21.15.111:0/1324914403 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f2560039650 0x7f256003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.449+0000 7f257d1d0700 1 --2- 172.21.15.111:0/1324914403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2578108550 0x7f2578197e60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:29.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.449+0000 7f257d1d0700 1 -- 172.21.15.111:0/1324914403 >> 172.21.15.111:0/1324914403 conn(0x7f25780fdf70 msgr2=0x7f25780fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:29.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.449+0000 7f257d1d0700 1 -- 172.21.15.111:0/1324914403 shutdown_connections 2024-04-25T11:35:29.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:29.449+0000 7f257d1d0700 1 -- 172.21.15.111:0/1324914403 wait complete. 2024-04-25T11:35:29.953 DEBUG:teuthology.orchestra.run.smithi094:mon.b> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T11:35:29.955 DEBUG:teuthology.orchestra.run.smithi111:mon.c> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T11:35:29.958 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T11:35:29.958 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T11:35:30.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:30 smithi083 ceph-mon[29330]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi094:172.21.15.94=b;smithi111:172.21.15.111=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:35:30.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:30 smithi083 ceph-mon[29330]: Saving service mon spec with placement smithi083:172.21.15.83=a;smithi094:172.21.15.94=b;smithi111:172.21.15.111=c;count:3 2024-04-25T11:35:30.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:30.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:35:30.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:30.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:35:30.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:30.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:35:30.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:30.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:30 smithi083 ceph-mon[29330]: Deploying daemon mon.c on smithi111 2024-04-25T11:35:30.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:30 smithi083 ceph-mon[29330]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:31.132 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:35:33.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:32 smithi083 ceph-mon[29330]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:33.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.766+0000 7efc265c1700 1 -- 172.21.15.111:0/1947817054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc20103340 msgr2=0x7efc20103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:33.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.766+0000 7efc265c1700 1 --2- 172.21.15.111:0/1947817054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc20103340 0x7efc20103720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efc10009910 tx=0x7efc10009c20 comp rx=0 tx=0).stop 2024-04-25T11:35:33.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.766+0000 7efc265c1700 1 -- 172.21.15.111:0/1947817054 shutdown_connections 2024-04-25T11:35:33.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.766+0000 7efc265c1700 1 --2- 172.21.15.111:0/1947817054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc20103340 0x7efc20103720 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:33.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.766+0000 7efc265c1700 1 -- 172.21.15.111:0/1947817054 >> 172.21.15.111:0/1947817054 conn(0x7efc20077ac0 msgr2=0x7efc20077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:33.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.766+0000 7efc265c1700 1 -- 172.21.15.111:0/1947817054 shutdown_connections 2024-04-25T11:35:33.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.766+0000 7efc265c1700 1 -- 172.21.15.111:0/1947817054 wait complete. 2024-04-25T11:35:33.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.767+0000 7efc265c1700 1 Processor -- start 2024-04-25T11:35:33.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.767+0000 7efc265c1700 1 -- start start 2024-04-25T11:35:33.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.767+0000 7efc265c1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc20103340 0x7efc2019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.767+0000 7efc265c1700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc2010ebb0 con 0x7efc20103340 2024-04-25T11:35:33.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.767+0000 7efc1ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc20103340 0x7efc2019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.767+0000 7efc1ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc20103340 0x7efc2019a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.111:59894/0 (socket says 172.21.15.111:59894) 2024-04-25T11:35:33.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.767+0000 7efc1ffff700 1 -- 172.21.15.111:0/1671505843 learned_addr learned my addr 172.21.15.111:0/1671505843 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-25T11:35:33.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.768+0000 7efc1ffff700 1 -- 172.21.15.111:0/1671505843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2019a600 con 0x7efc20103340 2024-04-25T11:35:33.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.768+0000 7efc1ffff700 1 --2- 172.21.15.111:0/1671505843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc20103340 0x7efc2019a0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efc100056d0 tx=0x7efc100057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.768+0000 7efc1d7fa700 1 -- 172.21.15.111:0/1671505843 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc10026070 con 0x7efc20103340 2024-04-25T11:35:33.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.768+0000 7efc1d7fa700 1 -- 172.21.15.111:0/1671505843 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc1000bc00 con 0x7efc20103340 2024-04-25T11:35:33.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.768+0000 7efc265c1700 1 -- 172.21.15.111:0/1671505843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2019a890 con 0x7efc20103340 2024-04-25T11:35:33.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.768+0000 7efc1d7fa700 1 -- 172.21.15.111:0/1671505843 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc10003920 con 0x7efc20103340 2024-04-25T11:35:33.758 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.768+0000 7efc265c1700 1 -- 172.21.15.111:0/1671505843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc2019acf0 con 0x7efc20103340 2024-04-25T11:35:33.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.769+0000 7efc265c1700 1 -- 172.21.15.111:0/1671505843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc200640a0 con 0x7efc20103340 2024-04-25T11:35:33.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.770+0000 7efc1d7fa700 1 -- 172.21.15.111:0/1671505843 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7efc10029070 con 0x7efc20103340 2024-04-25T11:35:33.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.770+0000 7efc1d7fa700 1 --2- 172.21.15.111:0/1671505843 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efc080396b0 0x7efc0803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:33.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.770+0000 7efc1d7fa700 1 -- 172.21.15.111:0/1671505843 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efc1001e070 con 0x7efc20103340 2024-04-25T11:35:33.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.770+0000 7efc1f7fe700 1 --2- 172.21.15.111:0/1671505843 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efc080396b0 0x7efc0803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:33.759 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.771+0000 7efc1f7fe700 1 --2- 172.21.15.111:0/1671505843 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efc080396b0 0x7efc0803bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efc14006fd0 tx=0x7efc14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:33.764 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:33.775+0000 7efc1d7fa700 1 -- 172.21.15.111:0/1671505843 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc10013070 con 0x7efc20103340 2024-04-25T11:35:34.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:34.073+0000 7efc265c1700 1 -- 172.21.15.111:0/1671505843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7efc201019a0 con 0x7efc20103340 2024-04-25T11:35:34.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:34.074+0000 7efc1d7fa700 1 -- 172.21.15.111:0/1671505843 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7efc10030020 con 0x7efc20103340 2024-04-25T11:35:34.062 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:35:34.062 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":1,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","modified":"2024-04-25T11:33:42.458778Z","created":"2024-04-25T11:33:42.458778Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T11:35:34.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:34.076+0000 7efc265c1700 1 -- 172.21.15.111:0/1671505843 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efc080396b0 msgr2=0x7efc0803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:34.076+0000 7efc265c1700 1 --2- 172.21.15.111:0/1671505843 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efc080396b0 0x7efc0803bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efc14006fd0 tx=0x7efc14006e40 comp rx=0 tx=0).stop 2024-04-25T11:35:34.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:34.076+0000 7efc265c1700 1 -- 172.21.15.111:0/1671505843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc20103340 msgr2=0x7efc2019a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:34.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:34.076+0000 7efc265c1700 1 --2- 172.21.15.111:0/1671505843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc20103340 0x7efc2019a0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efc100056d0 tx=0x7efc100057b0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:34.076+0000 7efc265c1700 1 -- 172.21.15.111:0/1671505843 shutdown_connections 2024-04-25T11:35:34.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:34.076+0000 7efc265c1700 1 --2- 172.21.15.111:0/1671505843 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7efc080396b0 0x7efc0803bb70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:34.076+0000 7efc265c1700 1 --2- 172.21.15.111:0/1671505843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc20103340 0x7efc2019a0c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:34.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:34.076+0000 7efc265c1700 1 -- 172.21.15.111:0/1671505843 >> 172.21.15.111:0/1671505843 conn(0x7efc20077ac0 msgr2=0x7efc2010a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:34.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:34.076+0000 7efc265c1700 1 -- 172.21.15.111:0/1671505843 shutdown_connections 2024-04-25T11:35:34.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:34.076+0000 7efc265c1700 1 -- 172.21.15.111:0/1671505843 wait complete. 2024-04-25T11:35:34.065 INFO:teuthology.orchestra.run.smithi111.stderr:dumped monmap epoch 1 2024-04-25T11:35:35.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:34 smithi083 ceph-mon[29330]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:35.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:34 smithi083 ceph-mon[29330]: from='client.? 172.21.15.111:0/1671505843' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:35:35.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:35 smithi111 systemd[1]: Started Ceph mon.c for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:35:35.709 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T11:35:35.709 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T11:35:35.941 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:35:40.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.331+0000 7f12718f5700 1 -- 172.21.15.111:0/1794467631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12540035c0 msgr2=0x7f1254005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.331+0000 7f12718f5700 1 --2- 172.21.15.111:0/1794467631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12540035c0 0x7f1254005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f126c1013a0 tx=0x7f125c00feb0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.331+0000 7f12718f5700 1 -- 172.21.15.111:0/1794467631 shutdown_connections 2024-04-25T11:35:40.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.331+0000 7f12718f5700 1 --2- 172.21.15.111:0/1794467631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12540035c0 0x7f1254005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.331+0000 7f12718f5700 1 -- 172.21.15.111:0/1794467631 >> 172.21.15.111:0/1794467631 conn(0x7f126c0fcf80 msgr2=0x7f126c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:40.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.331+0000 7f12718f5700 1 -- 172.21.15.111:0/1794467631 shutdown_connections 2024-04-25T11:35:40.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.331+0000 7f12718f5700 1 -- 172.21.15.111:0/1794467631 wait complete. 2024-04-25T11:35:40.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.332+0000 7f12718f5700 1 Processor -- start 2024-04-25T11:35:40.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.333+0000 7f12718f5700 1 -- start start 2024-04-25T11:35:40.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.333+0000 7f12718f5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12540035c0 0x7f126c103100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.333+0000 7f12718f5700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c103640 0x7f126c103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.333+0000 7f12718f5700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f126c1093c0 con 0x7f126c103640 2024-04-25T11:35:40.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.333+0000 7f12718f5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f126c109240 con 0x7f12540035c0 2024-04-25T11:35:40.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.333+0000 7f126affd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12540035c0 0x7f126c103100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:40.322 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.333+0000 7f126affd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12540035c0 0x7f126c103100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37940/0 (socket says 172.21.15.111:37940) 2024-04-25T11:35:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.333+0000 7f126affd700 1 -- 172.21.15.111:0/2006307596 learned_addr learned my addr 172.21.15.111:0/2006307596 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-25T11:35:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.333+0000 7f126affd700 1 -- 172.21.15.111:0/2006307596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12540035c0 msgr2=0x7f126c103100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T11:35:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.333+0000 7f126affd700 1 -- 172.21.15.111:0/2006307596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12540035c0 msgr2=0x7f126c103100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T11:35:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.333+0000 7f126affd700 1 --2- 172.21.15.111:0/2006307596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12540035c0 0x7f126c103100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:35:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.333+0000 7f126affd700 1 --2- 172.21.15.111:0/2006307596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12540035c0 0x7f126c103100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:35:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.333+0000 7f126a7fc700 1 --2- 172.21.15.111:0/2006307596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c103640 0x7f126c103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.334+0000 7f126a7fc700 1 -- 172.21.15.111:0/2006307596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12540035c0 msgr2=0x7f126c103100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:35:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.334+0000 7f126a7fc700 1 --2- 172.21.15.111:0/2006307596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12540035c0 0x7f126c103100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.334+0000 7f126a7fc700 1 -- 172.21.15.111:0/2006307596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f126c105020 con 0x7f126c103640 2024-04-25T11:35:40.323 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.334+0000 7f126a7fc700 1 --2- 172.21.15.111:0/2006307596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c103640 0x7f126c103ac0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f126000b4d0 tx=0x7f126000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:40.324 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.334+0000 7f12708f3700 1 -- 172.21.15.111:0/2006307596 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1260011630 con 0x7f126c103640 2024-04-25T11:35:40.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.334+0000 7f12708f3700 1 -- 172.21.15.111:0/2006307596 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1260011c80 con 0x7f126c103640 2024-04-25T11:35:40.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.334+0000 7f12718f5700 1 -- 172.21.15.111:0/2006307596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f126c1a42d0 con 0x7f126c103640 2024-04-25T11:35:40.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.334+0000 7f12708f3700 1 -- 172.21.15.111:0/2006307596 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1260010da0 con 0x7f126c103640 2024-04-25T11:35:40.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.334+0000 7f12718f5700 1 -- 172.21.15.111:0/2006307596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126c1a4810 con 0x7f126c103640 2024-04-25T11:35:40.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.335+0000 7f12708f3700 1 -- 172.21.15.111:0/2006307596 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1260018020 con 0x7f126c103640 2024-04-25T11:35:40.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.335+0000 7f12718f5700 1 -- 172.21.15.111:0/2006307596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f126c102bb0 con 0x7f126c103640 2024-04-25T11:35:40.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.335+0000 7f12708f3700 1 --2- 172.21.15.111:0/2006307596 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1258039440 0x7f125803b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:40.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.336+0000 7f12708f3700 1 -- 172.21.15.111:0/2006307596 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f126004c280 con 0x7f126c103640 2024-04-25T11:35:40.325 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.336+0000 7f126affd700 1 --2- 172.21.15.111:0/2006307596 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1258039440 0x7f125803b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:40.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.336+0000 7f126affd700 1 --2- 172.21.15.111:0/2006307596 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1258039440 0x7f125803b900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f125c00fa60 tx=0x7f125c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:40.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.341+0000 7f12708f3700 1 -- 172.21.15.111:0/2006307596 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f126001f020 con 0x7f126c103640 2024-04-25T11:35:40.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:35:40.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:40.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: mon.a calling monitor election 2024-04-25T11:35:40.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:40.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:40.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:40.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: mon.c calling monitor election 2024-04-25T11:35:40.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:40.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:40.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:40.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:40.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:40.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T11:35:40.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: monmap epoch 2 2024-04-25T11:35:40.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:35:40.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: last_changed 2024-04-25T11:35:35.190429+0000 2024-04-25T11:35:40.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: created 2024-04-25T11:33:42.458778+0000 2024-04-25T11:35:40.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: min_mon_release 19 (squid) 2024-04-25T11:35:40.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: election_strategy: 1 2024-04-25T11:35:40.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-25T11:35:40.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: 1: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.c 2024-04-25T11:35:40.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: fsmap 2024-04-25T11:35:40.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:35:40.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: mgrmap e13: x(active, since 50s) 2024-04-25T11:35:40.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: overall HEALTH_OK 2024-04-25T11:35:40.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:40.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:40.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:40.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:35:40.576 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:40.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.623+0000 7f12718f5700 1 -- 172.21.15.111:0/2006307596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f126c1a4bf0 con 0x7f126c103640 2024-04-25T11:35:40.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.624+0000 7f12708f3700 1 -- 172.21.15.111:0/2006307596 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f1260018570 con 0x7f126c103640 2024-04-25T11:35:40.612 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:35:40.612 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":2,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","modified":"2024-04-25T11:35:35.190429Z","created":"2024-04-25T11:33:42.458778Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T11:35:40.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.626+0000 7f12718f5700 1 -- 172.21.15.111:0/2006307596 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1258039440 msgr2=0x7f125803b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.626+0000 7f12718f5700 1 --2- 172.21.15.111:0/2006307596 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1258039440 0x7f125803b900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f125c00fa60 tx=0x7f125c005960 comp rx=0 tx=0).stop 2024-04-25T11:35:40.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.626+0000 7f12718f5700 1 -- 172.21.15.111:0/2006307596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c103640 msgr2=0x7f126c103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:40.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.626+0000 7f12718f5700 1 --2- 172.21.15.111:0/2006307596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c103640 0x7f126c103ac0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f126000b4d0 tx=0x7f126000b890 comp rx=0 tx=0).stop 2024-04-25T11:35:40.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.626+0000 7f12718f5700 1 -- 172.21.15.111:0/2006307596 shutdown_connections 2024-04-25T11:35:40.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.626+0000 7f12718f5700 1 --2- 172.21.15.111:0/2006307596 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1258039440 0x7f125803b900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.626+0000 7f12718f5700 1 --2- 172.21.15.111:0/2006307596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12540035c0 0x7f126c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.626+0000 7f12718f5700 1 --2- 172.21.15.111:0/2006307596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f126c103640 0x7f126c103ac0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:40.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.626+0000 7f12718f5700 1 -- 172.21.15.111:0/2006307596 >> 172.21.15.111:0/2006307596 conn(0x7f126c0fcf80 msgr2=0x7f126c06b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:40.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.626+0000 7f12718f5700 1 -- 172.21.15.111:0/2006307596 shutdown_connections 2024-04-25T11:35:40.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:40.626+0000 7f12718f5700 1 -- 172.21.15.111:0/2006307596 wait complete. 2024-04-25T11:35:40.616 INFO:teuthology.orchestra.run.smithi111.stderr:dumped monmap epoch 2 2024-04-25T11:35:40.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:35:40.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:40.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: mon.a calling monitor election 2024-04-25T11:35:40.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:40.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:40.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:40.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: mon.c calling monitor election 2024-04-25T11:35:40.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:40.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:40.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:40.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:40.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:40.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T11:35:40.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: monmap epoch 2 2024-04-25T11:35:40.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:35:40.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: last_changed 2024-04-25T11:35:35.190429+0000 2024-04-25T11:35:40.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: created 2024-04-25T11:33:42.458778+0000 2024-04-25T11:35:40.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: min_mon_release 19 (squid) 2024-04-25T11:35:40.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: election_strategy: 1 2024-04-25T11:35:40.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-25T11:35:40.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: 1: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.c 2024-04-25T11:35:40.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: fsmap 2024-04-25T11:35:40.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:35:40.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: mgrmap e13: x(active, since 50s) 2024-04-25T11:35:40.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: overall HEALTH_OK 2024-04-25T11:35:40.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:40.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:40.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:40.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:35:40.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:41.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:41 smithi111 ceph-mon[32784]: Deploying daemon mon.b on smithi094 2024-04-25T11:35:41.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:41 smithi111 ceph-mon[32784]: from='client.? 172.21.15.111:0/2006307596' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:35:41.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:41 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:41.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:41 smithi083 ceph-mon[29330]: Deploying daemon mon.b on smithi094 2024-04-25T11:35:41.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:41 smithi083 ceph-mon[29330]: from='client.? 172.21.15.111:0/2006307596' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:35:41.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:41 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:42.226 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T11:35:42.226 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T11:35:42.460 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:35:42.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:42 smithi111 ceph-mon[32784]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:42.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:42 smithi083 ceph-mon[29330]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:43.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.939+0000 7feeb2ce7700 1 -- 172.21.15.111:0/3992228674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeac100eb0 msgr2=0x7feeac101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:43.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.939+0000 7feeb2ce7700 1 --2- 172.21.15.111:0/3992228674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeac100eb0 0x7feeac101290 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00aa20 tx=0x7fee9c00ad30 comp rx=0 tx=0).stop 2024-04-25T11:35:43.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.939+0000 7feeb2ce7700 1 -- 172.21.15.111:0/3992228674 shutdown_connections 2024-04-25T11:35:43.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.939+0000 7feeb2ce7700 1 --2- 172.21.15.111:0/3992228674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeac100eb0 0x7feeac101290 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:43.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.939+0000 7feeb2ce7700 1 -- 172.21.15.111:0/3992228674 >> 172.21.15.111:0/3992228674 conn(0x7feeac077860 msgr2=0x7feeac077c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:43.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.939+0000 7feeb2ce7700 1 -- 172.21.15.111:0/3992228674 shutdown_connections 2024-04-25T11:35:43.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.939+0000 7feeb2ce7700 1 -- 172.21.15.111:0/3992228674 wait complete. 2024-04-25T11:35:43.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.940+0000 7feeb2ce7700 1 Processor -- start 2024-04-25T11:35:43.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.940+0000 7feeb2ce7700 1 -- start start 2024-04-25T11:35:43.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.940+0000 7feeb2ce7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeac100eb0 0x7feeac1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:43.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.940+0000 7feeb2ce7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeac1985f0 0x7feeac19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:43.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.940+0000 7feeb2ce7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feeac10a850 con 0x7feeac1985f0 2024-04-25T11:35:43.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.940+0000 7feeb2ce7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7feeac10a6d0 con 0x7feeac100eb0 2024-04-25T11:35:43.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.940+0000 7feeb0a83700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeac100eb0 0x7feeac1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:43.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.940+0000 7feeb0a83700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeac100eb0 0x7feeac1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37954/0 (socket says 172.21.15.111:37954) 2024-04-25T11:35:43.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.940+0000 7feeb0a83700 1 -- 172.21.15.111:0/1360556669 learned_addr learned my addr 172.21.15.111:0/1360556669 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-25T11:35:43.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.940+0000 7feeabfff700 1 --2- 172.21.15.111:0/1360556669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeac1985f0 0x7feeac19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:43.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.940+0000 7feeb0a83700 1 -- 172.21.15.111:0/1360556669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeac100eb0 msgr2=0x7feeac1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T11:35:43.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.940+0000 7feeb0a83700 1 -- 172.21.15.111:0/1360556669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeac100eb0 msgr2=0x7feeac1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T11:35:43.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.940+0000 7feeb0a83700 1 --2- 172.21.15.111:0/1360556669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeac100eb0 0x7feeac1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:35:43.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.940+0000 7feeb0a83700 1 --2- 172.21.15.111:0/1360556669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeac100eb0 0x7feeac1980b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:35:43.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.941+0000 7feeabfff700 1 -- 172.21.15.111:0/1360556669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeac100eb0 msgr2=0x7feeac1980b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:35:43.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.941+0000 7feeabfff700 1 --2- 172.21.15.111:0/1360556669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeac100eb0 0x7feeac1980b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:43.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.941+0000 7feeabfff700 1 -- 172.21.15.111:0/1360556669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeac198c30 con 0x7feeac1985f0 2024-04-25T11:35:43.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.941+0000 7feeabfff700 1 --2- 172.21.15.111:0/1360556669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeac1985f0 0x7feeac19c980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feea000b4d0 tx=0x7feea000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:43.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.941+0000 7feea9ffb700 1 -- 172.21.15.111:0/1360556669 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7feea0011630 con 0x7feeac1985f0 2024-04-25T11:35:43.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.941+0000 7feeb2ce7700 1 -- 172.21.15.111:0/1360556669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeac19cec0 con 0x7feeac1985f0 2024-04-25T11:35:43.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.941+0000 7feea9ffb700 1 -- 172.21.15.111:0/1360556669 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feea0011c80 con 0x7feeac1985f0 2024-04-25T11:35:43.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.941+0000 7feea9ffb700 1 -- 172.21.15.111:0/1360556669 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7feea0010da0 con 0x7feeac1985f0 2024-04-25T11:35:43.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.941+0000 7feeb2ce7700 1 -- 172.21.15.111:0/1360556669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeac19d420 con 0x7feeac1985f0 2024-04-25T11:35:43.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.942+0000 7feea9ffb700 1 -- 172.21.15.111:0/1360556669 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7feea0018020 con 0x7feeac1985f0 2024-04-25T11:35:43.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.942+0000 7feeb2ce7700 1 -- 172.21.15.111:0/1360556669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeac102340 con 0x7feeac1985f0 2024-04-25T11:35:43.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.942+0000 7feea9ffb700 1 --2- 172.21.15.111:0/1360556669 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fee940393f0 0x7fee9403b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:43.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.942+0000 7feea9ffb700 1 -- 172.21.15.111:0/1360556669 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7feea004c1b0 con 0x7feeac1985f0 2024-04-25T11:35:43.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.942+0000 7feeb0a83700 1 --2- 172.21.15.111:0/1360556669 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fee940393f0 0x7fee9403b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:43.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.943+0000 7feeb0a83700 1 --2- 172.21.15.111:0/1360556669 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fee940393f0 0x7fee9403b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00a860 tx=0x7fee9c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:43.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:43.948+0000 7feea9ffb700 1 -- 172.21.15.111:0/1360556669 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feea001f020 con 0x7feeac1985f0 2024-04-25T11:35:44.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:44.225+0000 7feea9ffb700 1 -- 172.21.15.111:0/1360556669 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea00177d0 con 0x7feeac1985f0 2024-04-25T11:35:44.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:44.227+0000 7feeb2ce7700 1 -- 172.21.15.111:0/1360556669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7feeac19d730 con 0x7feeac1985f0 2024-04-25T11:35:49.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:49.233+0000 7feea9ffb700 1 -- 172.21.15.111:0/1360556669 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7feea003e510 con 0x7feeac1985f0 2024-04-25T11:35:49.222 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-25T11:35:49.222 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":3,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","modified":"2024-04-25T11:35:44.225396Z","created":"2024-04-25T11:33:42.458778Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T11:35:49.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:49.237+0000 7feeb2ce7700 1 -- 172.21.15.111:0/1360556669 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fee940393f0 msgr2=0x7fee9403b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:49.237+0000 7feeb2ce7700 1 --2- 172.21.15.111:0/1360556669 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fee940393f0 0x7fee9403b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00a860 tx=0x7fee9c01a040 comp rx=0 tx=0).stop 2024-04-25T11:35:49.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:49.237+0000 7feeb2ce7700 1 -- 172.21.15.111:0/1360556669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeac1985f0 msgr2=0x7feeac19c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:49.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:49.237+0000 7feeb2ce7700 1 --2- 172.21.15.111:0/1360556669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeac1985f0 0x7feeac19c980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feea000b4d0 tx=0x7feea000b890 comp rx=0 tx=0).stop 2024-04-25T11:35:49.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:49.237+0000 7feeb2ce7700 1 -- 172.21.15.111:0/1360556669 shutdown_connections 2024-04-25T11:35:49.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:49.237+0000 7feeb2ce7700 1 --2- 172.21.15.111:0/1360556669 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fee940393f0 0x7fee9403b8b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:49.237+0000 7feeb2ce7700 1 --2- 172.21.15.111:0/1360556669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feeac100eb0 0x7feeac1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:49.237+0000 7feeb2ce7700 1 --2- 172.21.15.111:0/1360556669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feeac1985f0 0x7feeac19c980 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:49.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:49.237+0000 7feeb2ce7700 1 -- 172.21.15.111:0/1360556669 >> 172.21.15.111:0/1360556669 conn(0x7feeac077860 msgr2=0x7feeac104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:49.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:49.238+0000 7feeb2ce7700 1 -- 172.21.15.111:0/1360556669 shutdown_connections 2024-04-25T11:35:49.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:49.238+0000 7feeb2ce7700 1 -- 172.21.15.111:0/1360556669 wait complete. 2024-04-25T11:35:49.227 INFO:teuthology.orchestra.run.smithi111.stderr:dumped monmap epoch 3 2024-04-25T11:35:49.505 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:35:49.505 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: mon.a calling monitor election 2024-04-25T11:35:49.505 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: mon.c calling monitor election 2024-04-25T11:35:49.505 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:49.506 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: from='client.? 172.21.15.111:0/1360556669' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:35:49.506 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:49.506 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:49.506 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:49.506 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:49.506 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:49.506 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:49.506 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:49.507 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:49.507 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T11:35:49.507 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: monmap epoch 3 2024-04-25T11:35:49.507 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:35:49.507 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: last_changed 2024-04-25T11:35:44.225396+0000 2024-04-25T11:35:49.507 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: created 2024-04-25T11:33:42.458778+0000 2024-04-25T11:35:49.507 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: min_mon_release 19 (squid) 2024-04-25T11:35:49.507 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: election_strategy: 1 2024-04-25T11:35:49.507 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-25T11:35:49.508 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: 1: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.c 2024-04-25T11:35:49.508 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: 2: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-04-25T11:35:49.508 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: fsmap 2024-04-25T11:35:49.508 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:35:49.508 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: mgrmap e13: x(active, since 59s) 2024-04-25T11:35:49.508 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: overall HEALTH_OK 2024-04-25T11:35:49.508 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:49.508 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:49.508 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:49.508 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:49 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:49.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:35:49.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: mon.a calling monitor election 2024-04-25T11:35:49.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: mon.c calling monitor election 2024-04-25T11:35:49.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:49.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: from='client.? 172.21.15.111:0/1360556669' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T11:35:49.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:35:49.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:49.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:49.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:49.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:49.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:49.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:49.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:49.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T11:35:49.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: monmap epoch 3 2024-04-25T11:35:49.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:35:49.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: last_changed 2024-04-25T11:35:44.225396+0000 2024-04-25T11:35:49.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: created 2024-04-25T11:33:42.458778+0000 2024-04-25T11:35:49.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: min_mon_release 19 (squid) 2024-04-25T11:35:49.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: election_strategy: 1 2024-04-25T11:35:49.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-25T11:35:49.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: 1: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.c 2024-04-25T11:35:49.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: 2: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-04-25T11:35:49.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: fsmap 2024-04-25T11:35:49.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:35:49.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: mgrmap e13: x(active, since 59s) 2024-04-25T11:35:49.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: overall HEALTH_OK 2024-04-25T11:35:49.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:49.660 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:49.660 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:49.660 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:49.783 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T11:35:49.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T11:35:50.024 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:35:50.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:35:50.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:50 smithi111 ceph-mon[32784]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:50.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:50.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:50.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:35:50.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:50 smithi083 ceph-mon[29330]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:50.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:50.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:51.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:51 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:51.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.738+0000 7ffaaf02e700 1 -- 172.21.15.83:0/408327095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa81015b0 msgr2=0x7ffaa8101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.738+0000 7ffaaf02e700 1 --2- 172.21.15.83:0/408327095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa81015b0 0x7ffaa8101990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009910 tx=0x7ffa9c009c20 comp rx=0 tx=0).stop 2024-04-25T11:35:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.738+0000 7ffaaf02e700 1 -- 172.21.15.83:0/408327095 shutdown_connections 2024-04-25T11:35:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.738+0000 7ffaaf02e700 1 --2- 172.21.15.83:0/408327095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa81015b0 0x7ffaa8101990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.738+0000 7ffaaf02e700 1 -- 172.21.15.83:0/408327095 >> 172.21.15.83:0/408327095 conn(0x7ffaa80fce60 msgr2=0x7ffaa80ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.738+0000 7ffaaf02e700 1 -- 172.21.15.83:0/408327095 shutdown_connections 2024-04-25T11:35:51.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.738+0000 7ffaaf02e700 1 -- 172.21.15.83:0/408327095 wait complete. 2024-04-25T11:35:51.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.739+0000 7ffaaf02e700 1 Processor -- start 2024-04-25T11:35:51.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.739+0000 7ffaaf02e700 1 -- start start 2024-04-25T11:35:51.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.739+0000 7ffaaf02e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa81015b0 0x7ffaa8196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:51.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.739+0000 7ffaaf02e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffaa8197300 0x7ffaa8197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:51.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.739+0000 7ffaaf02e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffaa81a4430 0x7ffaa81a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:51.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.739+0000 7ffaaf02e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa810b830 con 0x7ffaa81015b0 2024-04-25T11:35:51.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.739+0000 7ffaaf02e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa810b6b0 con 0x7ffaa8197300 2024-04-25T11:35:51.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.739+0000 7ffaaf02e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa810b9b0 con 0x7ffaa81a4430 2024-04-25T11:35:51.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.739+0000 7ffaacdca700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa81015b0 0x7ffaa8196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:51.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.739+0000 7ffaacdca700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa81015b0 0x7ffaa8196dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53084/0 (socket says 172.21.15.83:53084) 2024-04-25T11:35:51.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.739+0000 7ffaacdca700 1 -- 172.21.15.83:0/1336558873 learned_addr learned my addr 172.21.15.83:0/1336558873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:35:51.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.740+0000 7ffaa7fff700 1 --2- 172.21.15.83:0/1336558873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffaa8197300 0x7ffaa8197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.740+0000 7ffaacdca700 1 -- 172.21.15.83:0/1336558873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffaa81a4430 msgr2=0x7ffaa81a6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.740+0000 7ffaacdca700 1 --2- 172.21.15.83:0/1336558873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffaa81a4430 0x7ffaa81a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.740+0000 7ffaacdca700 1 -- 172.21.15.83:0/1336558873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffaa8197300 msgr2=0x7ffaa8197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.740+0000 7ffaacdca700 1 --2- 172.21.15.83:0/1336558873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffaa8197300 0x7ffaa8197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.740+0000 7ffaacdca700 1 -- 172.21.15.83:0/1336558873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa81a6de0 con 0x7ffaa81015b0 2024-04-25T11:35:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.740+0000 7ffaa7fff700 1 --2- 172.21.15.83:0/1336558873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffaa8197300 0x7ffaa8197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:35:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.740+0000 7ffaacdca700 1 --2- 172.21.15.83:0/1336558873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa81015b0 0x7ffaa8196dc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c005b40 tx=0x7ffa9c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.740+0000 7ffaa5ffb700 1 -- 172.21.15.83:0/1336558873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c026070 con 0x7ffaa81015b0 2024-04-25T11:35:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.740+0000 7ffaa5ffb700 1 -- 172.21.15.83:0/1336558873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa9c00b6f0 con 0x7ffaa81015b0 2024-04-25T11:35:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.740+0000 7ffaaf02e700 1 -- 172.21.15.83:0/1336558873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa81a70d0 con 0x7ffaa81015b0 2024-04-25T11:35:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.740+0000 7ffaa5ffb700 1 -- 172.21.15.83:0/1336558873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c0038e0 con 0x7ffaa81015b0 2024-04-25T11:35:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.740+0000 7ffaaf02e700 1 -- 172.21.15.83:0/1336558873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa81a7610 con 0x7ffaa81015b0 2024-04-25T11:35:51.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.741+0000 7ffaa5ffb700 1 -- 172.21.15.83:0/1336558873 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ffa9c029070 con 0x7ffaa81015b0 2024-04-25T11:35:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.741+0000 7ffaa5ffb700 1 --2- 172.21.15.83:0/1336558873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ffa98039ae0 0x7ffa9803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.741+0000 7ffaa5ffb700 1 -- 172.21.15.83:0/1336558873 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ffa9c04e3e0 con 0x7ffaa81015b0 2024-04-25T11:35:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.742+0000 7ffaa7fff700 1 --2- 172.21.15.83:0/1336558873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ffa98039ae0 0x7ffa9803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.741+0000 7ffaaf02e700 1 -- 172.21.15.83:0/1336558873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa80640a0 con 0x7ffaa81015b0 2024-04-25T11:35:51.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.742+0000 7ffaa7fff700 1 --2- 172.21.15.83:0/1336558873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ffa98039ae0 0x7ffa9803bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009910 tx=0x7ffa94008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:51.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.747+0000 7ffaa5ffb700 1 -- 172.21.15.83:0/1336558873 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa9c030070 con 0x7ffaa81015b0 2024-04-25T11:35:51.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.960+0000 7ffaaf02e700 1 -- 172.21.15.83:0/1336558873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ffaa81a7870 con 0x7ffaa81015b0 2024-04-25T11:35:51.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.960+0000 7ffaa5ffb700 1 -- 172.21.15.83:0/1336558873 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7ffa9c013070 con 0x7ffaa81015b0 2024-04-25T11:35:51.949 INFO:teuthology.orchestra.run.smithi083.stdout:# minimal ceph.conf for 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:35:51.950 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-04-25T11:35:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:35:51.950 INFO:teuthology.orchestra.run.smithi083.stdout: mon_host = [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] 2024-04-25T11:35:51.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.962+0000 7ffaaf02e700 1 -- 172.21.15.83:0/1336558873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ffa98039ae0 msgr2=0x7ffa9803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:51.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.962+0000 7ffaaf02e700 1 --2- 172.21.15.83:0/1336558873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ffa98039ae0 0x7ffa9803bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009910 tx=0x7ffa94008040 comp rx=0 tx=0).stop 2024-04-25T11:35:51.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.962+0000 7ffaaf02e700 1 -- 172.21.15.83:0/1336558873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa81015b0 msgr2=0x7ffaa8196dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:51.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.962+0000 7ffaaf02e700 1 --2- 172.21.15.83:0/1336558873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa81015b0 0x7ffaa8196dc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c005b40 tx=0x7ffa9c005030 comp rx=0 tx=0).stop 2024-04-25T11:35:51.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.962+0000 7ffaaf02e700 1 -- 172.21.15.83:0/1336558873 shutdown_connections 2024-04-25T11:35:51.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.962+0000 7ffaaf02e700 1 --2- 172.21.15.83:0/1336558873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ffa98039ae0 0x7ffa9803bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:51.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.962+0000 7ffaaf02e700 1 --2- 172.21.15.83:0/1336558873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffaa81a4430 0x7ffaa81a6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:51.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.962+0000 7ffaaf02e700 1 --2- 172.21.15.83:0/1336558873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffaa81015b0 0x7ffaa8196dc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:51.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.962+0000 7ffaaf02e700 1 --2- 172.21.15.83:0/1336558873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffaa8197300 0x7ffaa8197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:51.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.962+0000 7ffaaf02e700 1 -- 172.21.15.83:0/1336558873 >> 172.21.15.83:0/1336558873 conn(0x7ffaa80fce60 msgr2=0x7ffaa8102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:51.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.962+0000 7ffaaf02e700 1 -- 172.21.15.83:0/1336558873 shutdown_connections 2024-04-25T11:35:51.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:35:51.962+0000 7ffaaf02e700 1 -- 172.21.15.83:0/1336558873 wait complete. 2024-04-25T11:35:52.268 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: mon.b calling monitor election 2024-04-25T11:35:52.268 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: mon.b calling monitor election 2024-04-25T11:35:52.268 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: mon.c calling monitor election 2024-04-25T11:35:52.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: mon.a calling monitor election 2024-04-25T11:35:52.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T11:35:52.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: monmap epoch 3 2024-04-25T11:35:52.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:35:52.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: last_changed 2024-04-25T11:35:44.225396+0000 2024-04-25T11:35:52.269 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: created 2024-04-25T11:33:42.458778+0000 2024-04-25T11:35:52.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: min_mon_release 19 (squid) 2024-04-25T11:35:52.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: election_strategy: 1 2024-04-25T11:35:52.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-25T11:35:52.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: 1: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.c 2024-04-25T11:35:52.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: 2: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-04-25T11:35:52.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: fsmap 2024-04-25T11:35:52.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:35:52.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: mgrmap e13: x(active, since 61s) 2024-04-25T11:35:52.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: overall HEALTH_OK 2024-04-25T11:35:52.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:52.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:52.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:52.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:52.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T11:35:52.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:35:52.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:35:52.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:52.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: Reconfiguring daemon mon.a on smithi083 2024-04-25T11:35:52.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:52.271 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:52 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1336558873' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:52.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: mon.b calling monitor election 2024-04-25T11:35:52.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: mon.b calling monitor election 2024-04-25T11:35:52.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: mon.c calling monitor election 2024-04-25T11:35:52.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: mon.a calling monitor election 2024-04-25T11:35:52.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T11:35:52.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: monmap epoch 3 2024-04-25T11:35:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:35:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: last_changed 2024-04-25T11:35:44.225396+0000 2024-04-25T11:35:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: created 2024-04-25T11:33:42.458778+0000 2024-04-25T11:35:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: min_mon_release 19 (squid) 2024-04-25T11:35:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: election_strategy: 1 2024-04-25T11:35:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-25T11:35:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: 1: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.c 2024-04-25T11:35:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: 2: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-04-25T11:35:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: fsmap 2024-04-25T11:35:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:35:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: mgrmap e13: x(active, since 61s) 2024-04-25T11:35:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: overall HEALTH_OK 2024-04-25T11:35:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:52.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T11:35:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:35:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:35:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: Reconfiguring daemon mon.a on smithi083 2024-04-25T11:35:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:52.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:52 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1336558873' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:52.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: mon.b calling monitor election 2024-04-25T11:35:52.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: mon.b calling monitor election 2024-04-25T11:35:52.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: mon.c calling monitor election 2024-04-25T11:35:52.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: mon.a calling monitor election 2024-04-25T11:35:52.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T11:35:52.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: monmap epoch 3 2024-04-25T11:35:52.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:35:52.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: last_changed 2024-04-25T11:35:44.225396+0000 2024-04-25T11:35:52.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: created 2024-04-25T11:33:42.458778+0000 2024-04-25T11:35:52.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: min_mon_release 19 (squid) 2024-04-25T11:35:52.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: election_strategy: 1 2024-04-25T11:35:52.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-25T11:35:52.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: 1: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.c 2024-04-25T11:35:52.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: 2: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.b 2024-04-25T11:35:52.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: fsmap 2024-04-25T11:35:52.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T11:35:52.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: mgrmap e13: x(active, since 61s) 2024-04-25T11:35:52.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: overall HEALTH_OK 2024-04-25T11:35:52.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:52.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:52.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:52.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:52.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T11:35:52.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:35:52.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:35:52.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:52.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: Reconfiguring daemon mon.a on smithi083 2024-04-25T11:35:52.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:52.575 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:52 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1336558873' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:52.947 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T11:35:52.947 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:35:52.947 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T11:35:52.974 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:35:52.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:35:53.042 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T11:35:53.042 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T11:35:53.073 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T11:35:53.073 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:35:53.141 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-25T11:35:53.141 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T11:35:53.173 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-25T11:35:53.173 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:35:53.239 INFO:tasks.cephadm:Adding mgr.x on smithi083 2024-04-25T11:35:53.239 INFO:tasks.cephadm:Adding mgr.y on smithi094 2024-04-25T11:35:53.239 INFO:tasks.cephadm:Adding mgr.z on smithi111 2024-04-25T11:35:53.239 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi083=x;smithi094=y;smithi111=z' 2024-04-25T11:35:53.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:53 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:53.510 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:35:53.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:53 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:53.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:53 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:35:54.415 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:54.415 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:54.415 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:54 smithi094 ceph-mon[33032]: Reconfiguring mon.b (monmap changed)... 2024-04-25T11:35:54.415 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:35:54.415 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:35:54.415 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:54.415 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:54 smithi094 ceph-mon[33032]: Reconfiguring daemon mon.b on smithi094 2024-04-25T11:35:54.415 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:54 smithi094 ceph-mon[33032]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:54.631 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:54.631 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:54.631 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:54 smithi111 ceph-mon[32784]: Reconfiguring mon.b (monmap changed)... 2024-04-25T11:35:54.631 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:35:54.631 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:35:54.631 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:54.631 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:54 smithi111 ceph-mon[32784]: Reconfiguring daemon mon.b on smithi094 2024-04-25T11:35:54.631 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:54 smithi111 ceph-mon[32784]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:54 smithi083 ceph-mon[29330]: Reconfiguring mon.b (monmap changed)... 2024-04-25T11:35:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:35:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:35:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:54 smithi083 ceph-mon[29330]: Reconfiguring daemon mon.b on smithi094 2024-04-25T11:35:54.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:54 smithi083 ceph-mon[29330]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:55.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.135+0000 7f427e53a700 1 -- 172.21.15.111:0/1395122235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42781030d0 msgr2=0x7f42781054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:55.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.135+0000 7f427e53a700 1 --2- 172.21.15.111:0/1395122235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42781030d0 0x7f42781054c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4268009910 tx=0x7f4268009c20 comp rx=0 tx=0).stop 2024-04-25T11:35:55.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.135+0000 7f427e53a700 1 -- 172.21.15.111:0/1395122235 shutdown_connections 2024-04-25T11:35:55.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.135+0000 7f427e53a700 1 --2- 172.21.15.111:0/1395122235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42781030d0 0x7f42781054c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.135+0000 7f427e53a700 1 -- 172.21.15.111:0/1395122235 >> 172.21.15.111:0/1395122235 conn(0x7f42780fca40 msgr2=0x7f42780fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:55.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.135+0000 7f427e53a700 1 -- 172.21.15.111:0/1395122235 shutdown_connections 2024-04-25T11:35:55.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.135+0000 7f427e53a700 1 -- 172.21.15.111:0/1395122235 wait complete. 2024-04-25T11:35:55.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.135+0000 7f427e53a700 1 Processor -- start 2024-04-25T11:35:55.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f427e53a700 1 -- start start 2024-04-25T11:35:55.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f427e53a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42781030d0 0x7f4278102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:55.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f427e53a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4278101080 0x7f4278101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:55.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f427e53a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278101a40 0x7f42781a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:55.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f427e53a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42781084e0 con 0x7f4278101080 2024-04-25T11:35:55.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f427e53a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4278108360 con 0x7f42781030d0 2024-04-25T11:35:55.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f427e53a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4278108660 con 0x7f4278101a40 2024-04-25T11:35:55.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f427cad7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278101a40 0x7f42781a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:55.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f427cad7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278101a40 0x7f42781a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34910/0 (socket says 172.21.15.111:34910) 2024-04-25T11:35:55.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f427cad7700 1 -- 172.21.15.111:0/3413194643 learned_addr learned my addr 172.21.15.111:0/3413194643 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-25T11:35:55.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f4277fff700 1 --2- 172.21.15.111:0/3413194643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42781030d0 0x7f4278102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:55.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f42777fe700 1 --2- 172.21.15.111:0/3413194643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4278101080 0x7f4278101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:55.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f427cad7700 1 -- 172.21.15.111:0/3413194643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42781030d0 msgr2=0x7f4278102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:55.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f427cad7700 1 --2- 172.21.15.111:0/3413194643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42781030d0 0x7f4278102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f427cad7700 1 -- 172.21.15.111:0/3413194643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4278101080 msgr2=0x7f4278101500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:55.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f427cad7700 1 --2- 172.21.15.111:0/3413194643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4278101080 0x7f4278101500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.136+0000 7f427cad7700 1 -- 172.21.15.111:0/3413194643 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42781a2310 con 0x7f4278101a40 2024-04-25T11:35:55.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.137+0000 7f427cad7700 1 --2- 172.21.15.111:0/3413194643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278101a40 0x7f42781a1d50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f427000ba50 tx=0x7f427000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:55.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.137+0000 7f42757fa700 1 -- 172.21.15.111:0/3413194643 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270019070 con 0x7f4278101a40 2024-04-25T11:35:55.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.137+0000 7f42757fa700 1 -- 172.21.15.111:0/3413194643 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4270014440 con 0x7f4278101a40 2024-04-25T11:35:55.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.137+0000 7f427e53a700 1 -- 172.21.15.111:0/3413194643 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42781a2600 con 0x7f4278101a40 2024-04-25T11:35:55.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.137+0000 7f42757fa700 1 -- 172.21.15.111:0/3413194643 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427000ce60 con 0x7f4278101a40 2024-04-25T11:35:55.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.137+0000 7f427e53a700 1 -- 172.21.15.111:0/3413194643 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42781a2c10 con 0x7f4278101a40 2024-04-25T11:35:55.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.138+0000 7f42757fa700 1 -- 172.21.15.111:0/3413194643 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f427001c070 con 0x7f4278101a40 2024-04-25T11:35:55.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.138+0000 7f427e53a700 1 -- 172.21.15.111:0/3413194643 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42780640a0 con 0x7f4278101a40 2024-04-25T11:35:55.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.138+0000 7f42757fa700 1 --2- 172.21.15.111:0/3413194643 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4260039ae0 0x7f426003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:35:55.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.138+0000 7f42757fa700 1 -- 172.21.15.111:0/3413194643 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f427004bdc0 con 0x7f4278101a40 2024-04-25T11:35:55.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.138+0000 7f4277fff700 1 --2- 172.21.15.111:0/3413194643 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4260039ae0 0x7f426003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:35:55.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.139+0000 7f4277fff700 1 --2- 172.21.15.111:0/3413194643 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4260039ae0 0x7f426003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4268005190 tx=0x7f426801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:35:55.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.143+0000 7f42757fa700 1 -- 172.21.15.111:0/3413194643 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f42700145f0 con 0x7f4278101a40 2024-04-25T11:35:55.347 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.359+0000 7f427e53a700 1 -- 172.21.15.111:0/3413194643 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi094=y;smithi111=z", "target": ["mon-mgr", ""]}) -- 0x7f4278062f10 con 0x7f4260039ae0 2024-04-25T11:35:55.354 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.366+0000 7f42757fa700 1 -- 172.21.15.111:0/3413194643 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4278062f10 con 0x7f4260039ae0 2024-04-25T11:35:55.354 INFO:teuthology.orchestra.run.smithi111.stdout:Scheduled mgr update... 2024-04-25T11:35:55.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.368+0000 7f427e53a700 1 -- 172.21.15.111:0/3413194643 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4260039ae0 msgr2=0x7f426003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:55.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.368+0000 7f427e53a700 1 --2- 172.21.15.111:0/3413194643 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4260039ae0 0x7f426003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4268005190 tx=0x7f426801a040 comp rx=0 tx=0).stop 2024-04-25T11:35:55.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.368+0000 7f427e53a700 1 -- 172.21.15.111:0/3413194643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278101a40 msgr2=0x7f42781a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:35:55.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.368+0000 7f427e53a700 1 --2- 172.21.15.111:0/3413194643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278101a40 0x7f42781a1d50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f427000ba50 tx=0x7f427000bd60 comp rx=0 tx=0).stop 2024-04-25T11:35:55.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.368+0000 7f427e53a700 1 -- 172.21.15.111:0/3413194643 shutdown_connections 2024-04-25T11:35:55.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.368+0000 7f427e53a700 1 --2- 172.21.15.111:0/3413194643 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4260039ae0 0x7f426003bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.368+0000 7f427e53a700 1 --2- 172.21.15.111:0/3413194643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4278101a40 0x7f42781a1d50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.368+0000 7f427e53a700 1 --2- 172.21.15.111:0/3413194643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42781030d0 0x7f4278102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.368+0000 7f427e53a700 1 --2- 172.21.15.111:0/3413194643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4278101080 0x7f4278101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:35:55.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.368+0000 7f427e53a700 1 -- 172.21.15.111:0/3413194643 >> 172.21.15.111:0/3413194643 conn(0x7f42780fca40 msgr2=0x7f42780fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:35:55.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.368+0000 7f427e53a700 1 -- 172.21.15.111:0/3413194643 shutdown_connections 2024-04-25T11:35:55.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:35:55.368+0000 7f427e53a700 1 -- 172.21.15.111:0/3413194643 wait complete. 2024-04-25T11:35:56.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:55 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:56.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:55 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:56.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:55 smithi083 ceph-mon[29330]: Reconfiguring mon.c (monmap changed)... 2024-04-25T11:35:56.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:55 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:35:56.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:55 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:35:56.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:55 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:56.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:55 smithi083 ceph-mon[29330]: Reconfiguring daemon mon.c on smithi111 2024-04-25T11:35:56.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:55 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:56.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:55 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:56.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:55 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:56.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:55 smithi094 ceph-mon[33032]: Reconfiguring mon.c (monmap changed)... 2024-04-25T11:35:56.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:55 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:35:56.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:55 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:35:56.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:55 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:56.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:55 smithi094 ceph-mon[33032]: Reconfiguring daemon mon.c on smithi111 2024-04-25T11:35:56.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:55 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:56.315 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:55 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:56.315 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:55 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:56.315 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:55 smithi111 ceph-mon[32784]: Reconfiguring mon.c (monmap changed)... 2024-04-25T11:35:56.315 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:55 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T11:35:56.316 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:55 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T11:35:56.316 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:55 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:56.316 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:55 smithi111 ceph-mon[32784]: Reconfiguring daemon mon.c on smithi111 2024-04-25T11:35:56.316 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:55 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:57.010 DEBUG:teuthology.orchestra.run.smithi094:mgr.y> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T11:35:57.013 DEBUG:teuthology.orchestra.run.smithi111:mgr.z> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T11:35:57.016 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T11:35:57.016 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:35:57.016 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:35:57.031 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:35:57.032 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-04-25T11:35:57.086 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:35:57.087 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:35:57.087 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T11:35:57.087 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:35:57.087 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:35:57.087 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-25 11:35:02.351188814 +0000 2024-04-25T11:35:57.087 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-25 11:29:52.038459605 +0000 2024-04-25T11:35:57.087 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-25 11:29:52.038459605 +0000 2024-04-25T11:35:57.087 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-25 11:29:52.038459605 +0000 2024-04-25T11:35:57.087 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:35:57.147 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:56 smithi083 ceph-mon[29330]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi094=y;smithi111=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:35:57.147 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:56 smithi083 ceph-mon[29330]: Saving service mgr spec with placement smithi083=x;smithi094=y;smithi111=z;count:3 2024-04-25T11:35:57.147 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:56 smithi083 ceph-mon[29330]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:57.147 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:56 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:57.147 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:56 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:57.147 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:56 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:35:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-25T11:35:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-25T11:35:57.150 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000211694 s, 2.4 MB/s 2024-04-25T11:35:57.152 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:35:57.207 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-04-25T11:35:57.262 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:35:57.263 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:35:57.263 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T11:35:57.263 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:35:57.263 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:35:57.263 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-25 11:35:02.378188134 +0000 2024-04-25T11:35:57.263 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-25 11:29:52.391450512 +0000 2024-04-25T11:35:57.263 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-25 11:29:52.391450512 +0000 2024-04-25T11:35:57.263 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-25 11:29:52.391450512 +0000 2024-04-25T11:35:57.264 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:35:57.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:56 smithi094 ceph-mon[33032]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi094=y;smithi111=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:35:57.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:56 smithi094 ceph-mon[33032]: Saving service mgr spec with placement smithi083=x;smithi094=y;smithi111=z;count:3 2024-04-25T11:35:57.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:56 smithi094 ceph-mon[33032]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:57.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:56 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:57.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:56 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:57.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:56 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:35:57.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:56 smithi111 ceph-mon[32784]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi094=y;smithi111=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:35:57.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:56 smithi111 ceph-mon[32784]: Saving service mgr spec with placement smithi083=x;smithi094=y;smithi111=z;count:3 2024-04-25T11:35:57.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:56 smithi111 ceph-mon[32784]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:57.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:56 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:57.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:56 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:57.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:56 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:35:57.328 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-25T11:35:57.328 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-25T11:35:57.328 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000211225 s, 2.4 MB/s 2024-04-25T11:35:57.329 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:35:57.385 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-04-25T11:35:57.439 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:35:57.439 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:35:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T11:35:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:35:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:35:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-25 11:35:02.403187505 +0000 2024-04-25T11:35:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-25 11:29:52.717442115 +0000 2024-04-25T11:35:57.440 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-25 11:29:52.717442115 +0000 2024-04-25T11:35:57.440 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-25 11:29:52.717442115 +0000 2024-04-25T11:35:57.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:35:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-25T11:35:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-25T11:35:57.501 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000230383 s, 2.2 MB/s 2024-04-25T11:35:57.503 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:35:57.558 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-04-25T11:35:57.613 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:35:57.613 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:35:57.613 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T11:35:57.613 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:35:57.613 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:35:57.613 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-25 11:35:02.429186850 +0000 2024-04-25T11:35:57.613 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-25 11:29:53.069433048 +0000 2024-04-25T11:35:57.613 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-25 11:29:53.069433048 +0000 2024-04-25T11:35:57.613 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-25 11:29:53.069433048 +0000 2024-04-25T11:35:57.613 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:35:57.674 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-25T11:35:57.674 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-25T11:35:57.674 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000215068 s, 2.4 MB/s 2024-04-25T11:35:57.675 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:35:57.733 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T11:35:57.733 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:35:57.752 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:35:57.752 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-04-25T11:35:57.810 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:35:57.810 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:35:57.810 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T11:35:57.810 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:35:57.810 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:35:57.810 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 11:35:43.661696873 +0000 2024-04-25T11:35:57.810 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 11:30:02.768728939 +0000 2024-04-25T11:35:57.811 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 11:30:02.768728939 +0000 2024-04-25T11:35:57.811 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 11:30:02.768728939 +0000 2024-04-25T11:35:57.811 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:35:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T11:35:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T11:35:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000116812 s, 4.4 MB/s 2024-04-25T11:35:57.878 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:35:57.936 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-04-25T11:35:57.993 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:35:57.993 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:35:57.994 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-25T11:35:57.994 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:35:57.994 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:35:57.994 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 11:35:43.661696873 +0000 2024-04-25T11:35:57.994 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 11:30:03.096723782 +0000 2024-04-25T11:35:57.994 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 11:30:03.096723782 +0000 2024-04-25T11:35:57.994 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 11:30:03.096723782 +0000 2024-04-25T11:35:57.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:35:58.059 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T11:35:58.059 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T11:35:58.059 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 9.8722e-05 s, 5.2 MB/s 2024-04-25T11:35:58.060 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:35:58.118 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-04-25T11:35:58.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:57 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:58.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:57 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:35:58.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:57 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:58.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:57 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:35:58.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:57 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:35:58.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:57 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:35:58.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:57 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:58.174 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:35:58.175 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:35:58.175 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T11:35:58.175 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:35:58.175 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:35:58.175 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 11:35:43.662696857 +0000 2024-04-25T11:35:58.175 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 11:30:03.441718357 +0000 2024-04-25T11:35:58.175 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 11:30:03.441718357 +0000 2024-04-25T11:35:58.175 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 11:30:03.441718357 +0000 2024-04-25T11:35:58.175 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:35:58.179 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:57 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:58.179 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:57 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:35:58.179 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:57 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:58.179 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:57 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:35:58.179 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:57 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:35:58.179 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:57 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:35:58.179 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:57 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:58.237 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:57 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:58.238 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:57 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:35:58.238 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:57 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:35:58.238 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:57 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:35:58.238 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:57 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:35:58.238 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:57 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:35:58.238 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:57 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:35:58.239 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T11:35:58.239 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T11:35:58.240 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000117846 s, 4.3 MB/s 2024-04-25T11:35:58.241 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:35:58.298 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-04-25T11:35:58.354 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:35:58.354 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:35:58.354 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T11:35:58.354 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:35:58.354 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:35:58.354 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 11:35:43.662696857 +0000 2024-04-25T11:35:58.354 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 11:30:03.766713248 +0000 2024-04-25T11:35:58.354 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 11:30:03.766713248 +0000 2024-04-25T11:35:58.354 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 11:30:03.766713248 +0000 2024-04-25T11:35:58.355 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:35:58.417 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T11:35:58.417 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T11:35:58.417 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000127155 s, 4.0 MB/s 2024-04-25T11:35:58.418 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:35:58.475 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-25T11:35:58.475 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:35:58.491 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:35:58.491 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2024-04-25T11:35:58.547 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:35:58.548 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:35:58.548 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T11:35:58.548 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:35:58.548 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:35:58.548 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-25 11:35:34.131377709 +0000 2024-04-25T11:35:58.548 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-25 11:30:01.489924873 +0000 2024-04-25T11:35:58.548 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-25 11:30:01.489924873 +0000 2024-04-25T11:35:58.548 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-25 11:30:01.489924873 +0000 2024-04-25T11:35:58.548 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:35:58.611 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-25T11:35:58.611 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-25T11:35:58.611 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000273517 s, 1.9 MB/s 2024-04-25T11:35:58.612 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:35:58.669 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2024-04-25T11:35:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:35:58.725 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:35:58.725 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T11:35:58.725 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:35:58.725 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:35:58.725 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-25 11:35:34.132377691 +0000 2024-04-25T11:35:58.726 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-25 11:30:01.831918442 +0000 2024-04-25T11:35:58.726 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-25 11:30:01.831918442 +0000 2024-04-25T11:35:58.726 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-25 11:30:01.831918442 +0000 2024-04-25T11:35:58.726 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:35:58.788 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-25T11:35:58.789 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-25T11:35:58.789 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000213853 s, 2.4 MB/s 2024-04-25T11:35:58.790 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:35:58.845 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2024-04-25T11:35:58.900 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:35:58.900 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:35:58.900 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T11:35:58.900 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:35:58.900 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:35:58.900 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-25 11:35:34.132377691 +0000 2024-04-25T11:35:58.900 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-25 11:30:02.166912141 +0000 2024-04-25T11:35:58.900 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-25 11:30:02.166912141 +0000 2024-04-25T11:35:58.900 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-25 11:30:02.166912141 +0000 2024-04-25T11:35:58.900 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:35:59.070 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-25T11:35:59.071 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-25T11:35:59.071 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000239708 s, 2.1 MB/s 2024-04-25T11:35:59.072 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:35:59.086 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2024-04-25T11:35:59.145 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:35:59.145 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:35:59.145 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T11:35:59.145 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:35:59.145 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:35:59.145 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-25 11:35:59.039936593 +0000 2024-04-25T11:35:59.145 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-25 11:30:02.504905783 +0000 2024-04-25T11:35:59.145 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-25 11:30:02.504905783 +0000 2024-04-25T11:35:59.145 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-25 11:30:02.504905783 +0000 2024-04-25T11:35:59.146 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:35:59.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:58 smithi083 ceph-mon[29330]: Deploying daemon mgr.z on smithi111 2024-04-25T11:35:59.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:35:58 smithi083 ceph-mon[29330]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:59.204 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:58 smithi111 ceph-mon[32784]: Deploying daemon mgr.z on smithi111 2024-04-25T11:35:59.204 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:35:58 smithi111 ceph-mon[32784]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:59.247 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-25T11:35:59.248 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-25T11:35:59.248 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000193468 s, 2.6 MB/s 2024-04-25T11:35:59.249 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:35:59.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:58 smithi094 ceph-mon[33032]: Deploying daemon mgr.z on smithi111 2024-04-25T11:35:59.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:35:58 smithi094 ceph-mon[33032]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:35:59.305 INFO:tasks.cephadm:Deploying osd.0 on smithi083 with /dev/vg_nvme/lv_4... 2024-04-25T11:35:59.305 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T11:35:59.485 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:35:59 smithi111 systemd[1]: Starting Ceph mgr.z for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:35:59.547 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:35:59.786 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:35:59 smithi111 podman[34612]: 2024-04-25 11:35:59.400332853 +0000 UTC m=+0.018203546 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:35:59.787 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:35:59 smithi111 podman[34612]: 2024-04-25 11:35:59.547444342 +0000 UTC m=+0.165315080 container create 3983775f5066404d461acf8727ef80295d6686cc64a278fdf6e242153dd2f77c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-25T11:35:59.787 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:35:59 smithi111 podman[34612]: 2024-04-25 11:35:59.799591348 +0000 UTC m=+0.417462040 container init 3983775f5066404d461acf8727ef80295d6686cc64a278fdf6e242153dd2f77c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:36:00.073 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:35:59 smithi111 podman[34612]: 2024-04-25 11:35:59.802533912 +0000 UTC m=+0.420404607 container start 3983775f5066404d461acf8727ef80295d6686cc64a278fdf6e242153dd2f77c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T11:36:00.073 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:35:59 smithi111 ceph-mgr[34631]: -- 172.21.15.111:0/904657274 <== mon.1 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564045c125a0 con 0x564045be7000 2024-04-25T11:36:00.073 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:35:59 smithi111 bash[34612]: 3983775f5066404d461acf8727ef80295d6686cc64a278fdf6e242153dd2f77c 2024-04-25T11:36:00.073 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:35:59 smithi111 systemd[1]: Started Ceph mgr.z for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:36:00.073 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:35:59 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:35:59.954+0000 7f4fbc2bd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:36:00.541 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:00 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:00.139+0000 7f4fbc2bd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:36:00.541 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:00 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:00.241+0000 7f4fbc2bd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:36:00.541 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:00 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:00.551+0000 7f4fbc2bd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:36:00.826 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:00 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:00.635+0000 7f4fbc2bd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:36:00.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:00 smithi083 ceph-mon[29330]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:00.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:00 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:00.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:00 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:00.961 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:00 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:00.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:00 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:36:00.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:00 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:36:00.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:00 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:36:00.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:00 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:00.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:00 smithi083 ceph-mon[29330]: Deploying daemon mgr.y on smithi094 2024-04-25T11:36:01.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:00 smithi094 ceph-mon[33032]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:01.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:00 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:01.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:00 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:01.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:00 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:01.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:00 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:36:01.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:00 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:36:01.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:00 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:36:01.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:00 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:01.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:00 smithi094 ceph-mon[33032]: Deploying daemon mgr.y on smithi094 2024-04-25T11:36:01.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:00 smithi111 ceph-mon[32784]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:01.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:00 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:01.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:00 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:01.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:00 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:01.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:00 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:36:01.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:00 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T11:36:01.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:00 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:36:01.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:00 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:01.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:00 smithi111 ceph-mon[32784]: Deploying daemon mgr.y on smithi094 2024-04-25T11:36:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T11:36:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T11:36:01.530 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-25T11:36:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-25T11:36:01.530 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116437 s, 901 MB/s 2024-04-25T11:36:01.530 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-25T11:36:02.017 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:01 smithi094 systemd[1]: Starting Ceph mgr.y for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:36:02.146 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_4 2024-04-25T11:36:02.174 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:01 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:01.837+0000 7f4fbc2bd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:36:02.174 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:02 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:02.011+0000 7f4fbc2bd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:36:02.174 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:02 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:02.186+0000 7f4fbc2bd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:36:02.270 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:02 smithi094 podman[34408]: 2024-04-25 11:36:01.933006052 +0000 UTC m=+0.018195039 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:36:02.270 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:02 smithi094 podman[34408]: 2024-04-25 11:36:02.08092735 +0000 UTC m=+0.166116352 container create 6b433dab8161c6375b436e7e3407f21d7c84384a14424bdaf9c32952dbdb7752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T11:36:02.270 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:02 smithi094 podman[34408]: 2024-04-25 11:36:02.283151991 +0000 UTC m=+0.368340955 container init 6b433dab8161c6375b436e7e3407f21d7c84384a14424bdaf9c32952dbdb7752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T11:36:02.388 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:36:02.524 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:02 smithi094 podman[34408]: 2024-04-25 11:36:02.285728282 +0000 UTC m=+0.370917244 container start 6b433dab8161c6375b436e7e3407f21d7c84384a14424bdaf9c32952dbdb7752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-25T11:36:02.524 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:02 smithi094 ceph-mgr[34427]: -- 172.21.15.94:0/201180122 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fffb5e05a0 con 0x55fffb5b5000 2024-04-25T11:36:02.524 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:02 smithi094 bash[34408]: 6b433dab8161c6375b436e7e3407f21d7c84384a14424bdaf9c32952dbdb7752 2024-04-25T11:36:02.524 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:02 smithi094 systemd[1]: Started Ceph mgr.y for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:36:02.524 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:02 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:02.446+0000 7f5ec94dd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:36:02.784 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:02 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:02.488+0000 7f4fbc2bd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:36:02.784 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:02 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:02.585+0000 7f4fbc2bd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:36:02.784 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:02 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:02.795+0000 7f4fbc2bd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:36:02.941 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:02 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:02.635+0000 7f5ec94dd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:36:02.941 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:02 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:02.740+0000 7f5ec94dd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:36:03.072 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:03 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:03.010+0000 7f4fbc2bd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:36:03.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:02 smithi111 ceph-mon[32784]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:03.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:02 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:03.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:02 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:03.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:02 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:03.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:02 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:03.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:02 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:03.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:02 smithi083 ceph-mon[29330]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:03.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:02 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:03.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:02 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:03.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:02 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:03.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:02 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:03.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:02 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:03.274 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:03 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:03.069+0000 7f5ec94dd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:36:03.274 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:03 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:03.175+0000 7f5ec94dd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:36:03.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:02 smithi094 ceph-mon[33032]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:03.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:02 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:03.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:02 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:03.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:02 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:03.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:02 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:03.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:02 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.777+0000 7f5160c41700 1 -- 172.21.15.83:0/3703575565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f515c1014b0 msgr2=0x7f515c10b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.777+0000 7f5160c41700 1 --2- 172.21.15.83:0/3703575565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f515c1014b0 0x7f515c10b080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f514c009990 tx=0x7f514c009ca0 comp rx=0 tx=0).stop 2024-04-25T11:36:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.777+0000 7f5160c41700 1 -- 172.21.15.83:0/3703575565 shutdown_connections 2024-04-25T11:36:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.777+0000 7f5160c41700 1 --2- 172.21.15.83:0/3703575565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f515c1014b0 0x7f515c10b080 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.777+0000 7f5160c41700 1 --2- 172.21.15.83:0/3703575565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f515c104090 0x7f515c1044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.777+0000 7f5160c41700 1 --2- 172.21.15.83:0/3703575565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f515c100b10 0x7f515c100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.777+0000 7f5160c41700 1 -- 172.21.15.83:0/3703575565 >> 172.21.15.83:0/3703575565 conn(0x7f515c0fc6b0 msgr2=0x7f515c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.778+0000 7f5160c41700 1 -- 172.21.15.83:0/3703575565 shutdown_connections 2024-04-25T11:36:03.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.778+0000 7f5160c41700 1 -- 172.21.15.83:0/3703575565 wait complete. 2024-04-25T11:36:03.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.778+0000 7f5160c41700 1 Processor -- start 2024-04-25T11:36:03.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.778+0000 7f5160c41700 1 -- start start 2024-04-25T11:36:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f5160c41700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f515c100b10 0x7f515c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f5160c41700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f515c1014b0 0x7f515c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f5160c41700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f515c104090 0x7f515c1a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f5160c41700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f515c10e550 con 0x7f515c100b10 2024-04-25T11:36:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f5160c41700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f515c10e3d0 con 0x7f515c1014b0 2024-04-25T11:36:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f5160c41700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f515c10e6d0 con 0x7f515c104090 2024-04-25T11:36:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f515a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f515c100b10 0x7f515c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f515a59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f515c100b10 0x7f515c198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55172/0 (socket says 172.21.15.83:55172) 2024-04-25T11:36:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f515a59c700 1 -- 172.21.15.83:0/3862595654 learned_addr learned my addr 172.21.15.83:0/3862595654 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:36:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f5159d9b700 1 --2- 172.21.15.83:0/3862595654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f515c1014b0 0x7f515c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f515ad9d700 1 --2- 172.21.15.83:0/3862595654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f515c104090 0x7f515c1a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f515a59c700 1 -- 172.21.15.83:0/3862595654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f515c104090 msgr2=0x7f515c1a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f515a59c700 1 --2- 172.21.15.83:0/3862595654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f515c104090 0x7f515c1a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f515a59c700 1 -- 172.21.15.83:0/3862595654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f515c1014b0 msgr2=0x7f515c199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:03.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f515a59c700 1 --2- 172.21.15.83:0/3862595654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f515c1014b0 0x7f515c199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f515a59c700 1 -- 172.21.15.83:0/3862595654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c1a07d0 con 0x7f515c100b10 2024-04-25T11:36:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f5159d9b700 1 --2- 172.21.15.83:0/3862595654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f515c1014b0 0x7f515c199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:36:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f515a59c700 1 --2- 172.21.15.83:0/3862595654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f515c100b10 0x7f515c198bc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5154007f40 tx=0x7f515400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f514b7fe700 1 -- 172.21.15.83:0/3862595654 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515400dd20 con 0x7f515c100b10 2024-04-25T11:36:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.779+0000 7f5160c41700 1 -- 172.21.15.83:0/3862595654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515c1a0ac0 con 0x7f515c100b10 2024-04-25T11:36:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.780+0000 7f514b7fe700 1 -- 172.21.15.83:0/3862595654 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5154018070 con 0x7f515c100b10 2024-04-25T11:36:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.780+0000 7f514b7fe700 1 -- 172.21.15.83:0/3862595654 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51540125b0 con 0x7f515c100b10 2024-04-25T11:36:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.780+0000 7f5160c41700 1 -- 172.21.15.83:0/3862595654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515c1a0fb0 con 0x7f515c100b10 2024-04-25T11:36:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.780+0000 7f514b7fe700 1 -- 172.21.15.83:0/3862595654 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5154012750 con 0x7f515c100b10 2024-04-25T11:36:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.781+0000 7f514b7fe700 1 --2- 172.21.15.83:0/3862595654 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5144039890 0x7f514403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:03.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.781+0000 7f5160c41700 1 -- 172.21.15.83:0/3862595654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515c069730 con 0x7f515c100b10 2024-04-25T11:36:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.781+0000 7f514b7fe700 1 -- 172.21.15.83:0/3862595654 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f515404d6d0 con 0x7f515c100b10 2024-04-25T11:36:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.781+0000 7f5159d9b700 1 --2- 172.21.15.83:0/3862595654 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5144039890 0x7f514403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:03.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.781+0000 7f5159d9b700 1 --2- 172.21.15.83:0/3862595654 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5144039890 0x7f514403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5150011490 tx=0x7f5150009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:03.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:03.786+0000 7f514b7fe700 1 -- 172.21.15.83:0/3862595654 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5154010070 con 0x7f515c100b10 2024-04-25T11:36:03.822 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:03 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:03.639+0000 7f4fbc2bd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:36:03.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:04.005+0000 7f5160c41700 1 -- 172.21.15.83:0/3862595654 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f515c062f40 con 0x7f5144039890 2024-04-25T11:36:04.323 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:04.066+0000 7f4fbc2bd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:36:04.323 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:04.161+0000 7f4fbc2bd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:36:04.723 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:04.398+0000 7f5ec94dd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:36:04.723 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:04.567+0000 7f5ec94dd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:36:04.724 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:04.734+0000 7f5ec94dd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:36:04.822 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:04.359+0000 7f4fbc2bd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:36:05.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:05.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:05.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:05.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:05.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:05.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:05.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:05.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:05.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T11:36:05.009 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:36:05.009 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:36:05.009 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:05.009 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: Reconfiguring daemon mgr.x on smithi083 2024-04-25T11:36:05.009 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:36:05.009 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:36:05.009 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:36:05.009 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:05.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:05.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:05.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:05.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:05.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:05.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:05.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:05.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:05.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T11:36:05.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:36:05.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:36:05.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:05.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: Reconfiguring daemon mgr.x on smithi083 2024-04-25T11:36:05.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:36:05.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:36:05.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:36:05.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:05.186 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:04.872+0000 7f4fbc2bd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:36:05.186 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:04.962+0000 7f4fbc2bd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:36:05.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:05.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:05.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:05.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:05.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:05.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:05.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:05.187 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:05.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T11:36:05.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T11:36:05.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T11:36:05.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:05.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: Reconfiguring daemon mgr.x on smithi083 2024-04-25T11:36:05.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:36:05.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:36:05.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:36:05.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:05.273 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:05 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:05.019+0000 7f5ec94dd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:36:05.274 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:05 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:05.110+0000 7f5ec94dd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:36:05.523 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:05 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:05.198+0000 7f4fbc2bd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:36:05.523 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:05 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:05.329+0000 7f4fbc2bd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:36:05.523 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:05 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:05.422+0000 7f4fbc2bd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:36:05.523 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:05 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:05.534+0000 7f4fbc2bd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:36:05.773 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:05 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:05.310+0000 7f5ec94dd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:36:05.774 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:05 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:05.500+0000 7f5ec94dd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:36:06.072 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:05 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:05.832+0000 7f4fbc2bd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:36:06.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:05 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:06.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:05 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:06.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:05 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:06.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:05 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:06.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:05 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:06.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:05 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:06.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:05 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:06.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:05 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:06.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:05 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:06.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:05 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:06.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:05 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:06.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:05 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:06.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:05 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:06.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:05 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:06.273 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:06 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:06.096+0000 7f5ec94dd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:36:06.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:05 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:06.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:05 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:06.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:05 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:06.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:05 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:06.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:05 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:06.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:05 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:06.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:05 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:06.773 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:06 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:06.510+0000 7f5ec94dd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:36:06.774 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:06 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:06.605+0000 7f5ec94dd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:36:06.823 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:06 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:06.431+0000 7f4fbc2bd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:36:06.823 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:36:06 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:36:06.645+0000 7f4fbc2bd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:36:06.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:06.892+0000 7f514b7fe700 1 -- 172.21.15.83:0/3862595654 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f515401d980 con 0x7f515c100b10 2024-04-25T11:36:06.968 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:06 smithi083 ceph-mon[29330]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:06.969 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:06 smithi083 ceph-mon[29330]: Standby manager daemon z started 2024-04-25T11:36:06.969 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:06 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.111:0/3618922080' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T11:36:06.969 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:06 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.111:0/3618922080' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:36:06.969 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:06 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.111:0/3618922080' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T11:36:06.969 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:06 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.111:0/3618922080' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:36:07.273 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:06 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:06.805+0000 7f5ec94dd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:36:07.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:06 smithi094 ceph-mon[33032]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:07.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:06 smithi094 ceph-mon[33032]: Standby manager daemon z started 2024-04-25T11:36:07.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:06 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.111:0/3618922080' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T11:36:07.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:06 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.111:0/3618922080' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:36:07.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:06 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.111:0/3618922080' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T11:36:07.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:06 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.111:0/3618922080' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:36:07.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:06 smithi111 ceph-mon[32784]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:07.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:06 smithi111 ceph-mon[32784]: Standby manager daemon z started 2024-04-25T11:36:07.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:06 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.111:0/3618922080' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T11:36:07.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:06 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.111:0/3618922080' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:36:07.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:06 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.111:0/3618922080' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T11:36:07.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:06 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.111:0/3618922080' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:36:07.637 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:07 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:07.319+0000 7f5ec94dd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:36:07.637 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:07 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:07.410+0000 7f5ec94dd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:36:07.977 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:07 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:07.649+0000 7f5ec94dd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:36:07.977 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:07 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:07.780+0000 7f5ec94dd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:36:07.977 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:07 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:07.873+0000 7f5ec94dd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:36:07.977 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:07 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:07.988+0000 7f5ec94dd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:36:07.977 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:07 smithi094 ceph-mon[33032]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T11:36:07.977 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:07 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T11:36:07.977 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:07 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/4065416431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a79d318f-21ba-4f1b-a80c-54a8fa4618b0"}]: dispatch 2024-04-25T11:36:07.978 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:07 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/4065416431' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a79d318f-21ba-4f1b-a80c-54a8fa4618b0"}]': finished 2024-04-25T11:36:07.978 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:07 smithi094 ceph-mon[33032]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T11:36:07.978 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:07 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:36:08.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:07 smithi083 ceph-mon[29330]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T11:36:08.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:07 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T11:36:08.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:07 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/4065416431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a79d318f-21ba-4f1b-a80c-54a8fa4618b0"}]: dispatch 2024-04-25T11:36:08.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:07 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/4065416431' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a79d318f-21ba-4f1b-a80c-54a8fa4618b0"}]': finished 2024-04-25T11:36:08.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:07 smithi083 ceph-mon[29330]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T11:36:08.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:07 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:36:08.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:07 smithi111 ceph-mon[32784]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T11:36:08.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:07 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T11:36:08.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:07 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/4065416431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a79d318f-21ba-4f1b-a80c-54a8fa4618b0"}]: dispatch 2024-04-25T11:36:08.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:07 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/4065416431' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a79d318f-21ba-4f1b-a80c-54a8fa4618b0"}]': finished 2024-04-25T11:36:08.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:07 smithi111 ceph-mon[32784]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T11:36:08.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:07 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:36:08.773 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:08 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:08.289+0000 7f5ec94dd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:36:09.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:08 smithi083 ceph-mon[29330]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:09.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:08 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/627589594' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:36:09.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:08 smithi094 ceph-mon[33032]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:09.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:08 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/627589594' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:36:09.274 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:08 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:08.881+0000 7f5ec94dd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:36:09.274 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:36:09 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:36:09.095+0000 7f5ec94dd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:36:09.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:08 smithi111 ceph-mon[32784]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:09.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:08 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/627589594' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:36:09.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:09.904+0000 7f514b7fe700 1 -- 172.21.15.83:0/3862595654 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f51540112b0 con 0x7f515c100b10 2024-04-25T11:36:10.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:09 smithi083 ceph-mon[29330]: Standby manager daemon y started 2024-04-25T11:36:10.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:09 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.94:0/2464110297' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T11:36:10.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:09 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.94:0/2464110297' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:36:10.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:09 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.94:0/2464110297' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T11:36:10.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:09 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.94:0/2464110297' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:36:10.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:09 smithi094 ceph-mon[33032]: Standby manager daemon y started 2024-04-25T11:36:10.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:09 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.94:0/2464110297' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T11:36:10.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:09 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.94:0/2464110297' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:36:10.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:09 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.94:0/2464110297' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T11:36:10.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:09 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.94:0/2464110297' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:36:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:09 smithi111 ceph-mon[32784]: Standby manager daemon y started 2024-04-25T11:36:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:09 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.94:0/2464110297' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T11:36:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:09 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.94:0/2464110297' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:36:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:09 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.94:0/2464110297' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T11:36:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:09 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.94:0/2464110297' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:36:11.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:10 smithi083 ceph-mon[29330]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:11.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:10 smithi083 ceph-mon[29330]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-25T11:36:11.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T11:36:11.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:10 smithi094 ceph-mon[33032]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:11.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:10 smithi094 ceph-mon[33032]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-25T11:36:11.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T11:36:11.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:10 smithi111 ceph-mon[32784]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:11.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:10 smithi111 ceph-mon[32784]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-25T11:36:11.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T11:36:13.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:12 smithi111 ceph-mon[32784]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:13.118 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:12 smithi083 ceph-mon[29330]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:13.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:12 smithi094 ceph-mon[33032]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:15.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:14 smithi083 ceph-mon[29330]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:15.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:14 smithi094 ceph-mon[33032]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:15.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:14 smithi111 ceph-mon[32784]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:15.981 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:15 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T11:36:15.981 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:15 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:16.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:15 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T11:36:16.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:15 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:16.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:15 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T11:36:16.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:15 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:17.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:16 smithi083 ceph-mon[29330]: Deploying daemon osd.0 on smithi083 2024-04-25T11:36:17.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:16 smithi083 ceph-mon[29330]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:17.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:16 smithi094 ceph-mon[33032]: Deploying daemon osd.0 on smithi083 2024-04-25T11:36:17.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:16 smithi094 ceph-mon[33032]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:17.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:16 smithi111 ceph-mon[32784]: Deploying daemon osd.0 on smithi083 2024-04-25T11:36:17.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:16 smithi111 ceph-mon[32784]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:19.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:18 smithi083 ceph-mon[29330]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:19.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:18 smithi094 ceph-mon[33032]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:19.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:18 smithi111 ceph-mon[32784]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:21.136 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:20 smithi083 ceph-mon[29330]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:21.137 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:21.137 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:21.137 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:21.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:20 smithi094 ceph-mon[33032]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:21.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:21.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:21.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:21.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:20 smithi111 ceph-mon[32784]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:21.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:21.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:21.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:23.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:22 smithi083 ceph-mon[29330]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:23.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:22 smithi094 ceph-mon[33032]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:23.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:22 smithi111 ceph-mon[32784]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:24.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:24 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:24.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:24 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:24.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:24 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:24.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:24 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:24.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:24 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:24.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:24 smithi083 ceph-mon[29330]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:24.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:24.227+0000 7f514b7fe700 1 -- 172.21.15.83:0/3862595654 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f515c062f40 con 0x7f5144039890 2024-04-25T11:36:24.216 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 0 on host 'smithi083' 2024-04-25T11:36:24.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:24.230+0000 7f5160c41700 1 -- 172.21.15.83:0/3862595654 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5144039890 msgr2=0x7f514403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:24.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:24.230+0000 7f5160c41700 1 --2- 172.21.15.83:0/3862595654 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5144039890 0x7f514403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5150011490 tx=0x7f5150009260 comp rx=0 tx=0).stop 2024-04-25T11:36:24.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:24.230+0000 7f5160c41700 1 -- 172.21.15.83:0/3862595654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f515c100b10 msgr2=0x7f515c198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:24.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:24.230+0000 7f5160c41700 1 --2- 172.21.15.83:0/3862595654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f515c100b10 0x7f515c198bc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5154007f40 tx=0x7f515400d3b0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:24.230+0000 7f5160c41700 1 -- 172.21.15.83:0/3862595654 shutdown_connections 2024-04-25T11:36:24.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:24.230+0000 7f5160c41700 1 --2- 172.21.15.83:0/3862595654 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5144039890 0x7f514403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:24.230+0000 7f5160c41700 1 --2- 172.21.15.83:0/3862595654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f515c104090 0x7f515c1a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:24.230+0000 7f5160c41700 1 --2- 172.21.15.83:0/3862595654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f515c100b10 0x7f515c198bc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:24.230+0000 7f5160c41700 1 --2- 172.21.15.83:0/3862595654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f515c1014b0 0x7f515c199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:24.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:24.230+0000 7f5160c41700 1 -- 172.21.15.83:0/3862595654 >> 172.21.15.83:0/3862595654 conn(0x7f515c0fc6b0 msgr2=0x7f515c1071b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:24.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:24.230+0000 7f5160c41700 1 -- 172.21.15.83:0/3862595654 shutdown_connections 2024-04-25T11:36:24.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:36:24.230+0000 7f5160c41700 1 -- 172.21.15.83:0/3862595654 wait complete. 2024-04-25T11:36:24.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:24 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:24.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:24 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:24.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:24 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:24.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:24 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:24.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:24 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:24.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:24 smithi111 ceph-mon[32784]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:24.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:24 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:24.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:24 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:24.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:24 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:24.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:24 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:24.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:24 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:24.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:24 smithi094 ceph-mon[33032]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:24.889 DEBUG:teuthology.orchestra.run.smithi083:osd.0> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T11:36:24.891 INFO:tasks.cephadm:Deploying osd.1 on smithi094 with /dev/vg_nvme/lv_4... 2024-04-25T11:36:24.891 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T11:36:25.137 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:36:25.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:25 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:25.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:25 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:25.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:25 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:25.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:25 smithi083 ceph-mon[29330]: from='osd.0 [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T11:36:25.453 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:25 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:25.454 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:25 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:25.454 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:25 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:25.454 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:25 smithi094 ceph-mon[33032]: from='osd.0 [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T11:36:25.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:25 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:25.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:25 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:25.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:25 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:25.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:25 smithi111 ceph-mon[32784]: from='osd.0 [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T11:36:26.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:26 smithi083 ceph-mon[29330]: from='osd.0 [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T11:36:26.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:26 smithi083 ceph-mon[29330]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T11:36:26.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:26 smithi083 ceph-mon[29330]: from='osd.0 [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T11:36:26.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:26 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:36:26.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:26 smithi083 ceph-mon[29330]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:26.320 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:36:26 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-0[44082]: 2024-04-25T11:36:26.217+0000 7fe4dce4e700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T11:36:26.320 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:36:26 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-0[44082]: 2024-04-25T11:36:26.221+0000 7fe4d8c41700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:36:26.458 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:26 smithi094 ceph-mon[33032]: from='osd.0 [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T11:36:26.458 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:26 smithi094 ceph-mon[33032]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T11:36:26.459 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:26 smithi094 ceph-mon[33032]: from='osd.0 [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T11:36:26.459 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:26 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:36:26.459 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:26 smithi094 ceph-mon[33032]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:26.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:26 smithi111 ceph-mon[32784]: from='osd.0 [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T11:36:26.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:26 smithi111 ceph-mon[32784]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T11:36:26.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:26 smithi111 ceph-mon[32784]: from='osd.0 [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T11:36:26.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:26 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:36:26.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:26 smithi111 ceph-mon[32784]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:27.001 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T11:36:27.001 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T11:36:27.002 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-25T11:36:27.002 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-25T11:36:27.002 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00932363 s, 1.1 GB/s 2024-04-25T11:36:27.002 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-25T11:36:27.307 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:27 smithi094 ceph-mon[33032]: from='osd.0 [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T11:36:27.307 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:27 smithi094 ceph-mon[33032]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T11:36:27.307 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:27 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:36:27.307 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:27 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:36:27.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:27 smithi083 ceph-mon[29330]: from='osd.0 [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T11:36:27.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:27 smithi083 ceph-mon[29330]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T11:36:27.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:27 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:36:27.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:27 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:36:27.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:27 smithi111 ceph-mon[32784]: from='osd.0 [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T11:36:27.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:27 smithi111 ceph-mon[32784]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T11:36:27.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:27 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:36:27.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:27 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:36:27.609 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_4 2024-04-25T11:36:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:36:28.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:28 smithi094 ceph-mon[33032]: purged_snaps scrub starts 2024-04-25T11:36:28.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:28 smithi094 ceph-mon[33032]: purged_snaps scrub ok 2024-04-25T11:36:28.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:28 smithi094 ceph-mon[33032]: osd.0 [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106] boot 2024-04-25T11:36:28.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:28 smithi094 ceph-mon[33032]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T11:36:28.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:28 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:36:28.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:28 smithi094 ceph-mon[33032]: Detected new or changed devices on smithi083 2024-04-25T11:36:28.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:28 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:28.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:28 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:28.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:28 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:36:28.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:28 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi083 to 17088M 2024-04-25T11:36:28.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:28 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:28.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:28 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:28.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:28 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:28.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:28 smithi094 ceph-mon[33032]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:28.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:28 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:28.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:28 smithi111 ceph-mon[32784]: purged_snaps scrub starts 2024-04-25T11:36:28.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:28 smithi111 ceph-mon[32784]: purged_snaps scrub ok 2024-04-25T11:36:28.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:28 smithi111 ceph-mon[32784]: osd.0 [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106] boot 2024-04-25T11:36:28.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:28 smithi111 ceph-mon[32784]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T11:36:28.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:28 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:36:28.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:28 smithi111 ceph-mon[32784]: Detected new or changed devices on smithi083 2024-04-25T11:36:28.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:28 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:28.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:28 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:28.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:28 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:36:28.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:28 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi083 to 17088M 2024-04-25T11:36:28.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:28 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:28.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:28 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:28.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:28 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:28.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:28 smithi111 ceph-mon[32784]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:28.574 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:28 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:28.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:28 smithi083 ceph-mon[29330]: purged_snaps scrub starts 2024-04-25T11:36:28.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:28 smithi083 ceph-mon[29330]: purged_snaps scrub ok 2024-04-25T11:36:28.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:28 smithi083 ceph-mon[29330]: osd.0 [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106] boot 2024-04-25T11:36:28.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:28 smithi083 ceph-mon[29330]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T11:36:28.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:28 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:36:28.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:28 smithi083 ceph-mon[29330]: Detected new or changed devices on smithi083 2024-04-25T11:36:28.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:28 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:28.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:28 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:28.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:28 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:36:28.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:28 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi083 to 17088M 2024-04-25T11:36:28.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:28 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:28.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:28 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:28.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:28 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:28.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:28 smithi083 ceph-mon[29330]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T11:36:28.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:28 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:29.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.441+0000 7f748f611700 1 -- 172.21.15.94:0/188811949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7488102cf0 msgr2=0x7f74881095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:29.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.441+0000 7f748f611700 1 --2- 172.21.15.94:0/188811949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7488102cf0 0x7f74881095a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f748000b0d0 tx=0x7f748000b3e0 comp rx=0 tx=0).stop 2024-04-25T11:36:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.441+0000 7f748f611700 1 -- 172.21.15.94:0/188811949 shutdown_connections 2024-04-25T11:36:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.441+0000 7f748f611700 1 --2- 172.21.15.94:0/188811949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7488102cf0 0x7f74881095a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.441+0000 7f748f611700 1 --2- 172.21.15.94:0/188811949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7488101150 0x7f7488101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.441+0000 7f748f611700 1 --2- 172.21.15.94:0/188811949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7488102350 0x7f74881027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.441+0000 7f748f611700 1 -- 172.21.15.94:0/188811949 >> 172.21.15.94:0/188811949 conn(0x7f74880fc6f0 msgr2=0x7f74880feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.441+0000 7f748f611700 1 -- 172.21.15.94:0/188811949 shutdown_connections 2024-04-25T11:36:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.441+0000 7f748f611700 1 -- 172.21.15.94:0/188811949 wait complete. 2024-04-25T11:36:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.441+0000 7f748f611700 1 Processor -- start 2024-04-25T11:36:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748f611700 1 -- start start 2024-04-25T11:36:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748f611700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7488101150 0x7f7488198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748f611700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7488102350 0x7f7488199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748f611700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7488102cf0 0x7f74881a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748f611700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f748810cc50 con 0x7f7488102350 2024-04-25T11:36:29.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748f611700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f748810cad0 con 0x7f7488102cf0 2024-04-25T11:36:29.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748f611700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f748810cdd0 con 0x7f7488101150 2024-04-25T11:36:29.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748dbae700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7488102cf0 0x7f74881a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:29.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748dbae700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7488102cf0 0x7f74881a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53718/0 (socket says 172.21.15.94:53718) 2024-04-25T11:36:29.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748dbae700 1 -- 172.21.15.94:0/101375268 learned_addr learned my addr 172.21.15.94:0/101375268 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T11:36:29.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748cbac700 1 --2- 172.21.15.94:0/101375268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7488102350 0x7f7488199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:29.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748d3ad700 1 --2- 172.21.15.94:0/101375268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7488101150 0x7f7488198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:29.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748dbae700 1 -- 172.21.15.94:0/101375268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7488101150 msgr2=0x7f7488198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:29.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748dbae700 1 --2- 172.21.15.94:0/101375268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7488101150 0x7f7488198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:29.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748dbae700 1 -- 172.21.15.94:0/101375268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7488102350 msgr2=0x7f7488199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:29.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748dbae700 1 --2- 172.21.15.94:0/101375268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7488102350 0x7f7488199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:29.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748dbae700 1 -- 172.21.15.94:0/101375268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74881a07d0 con 0x7f7488102cf0 2024-04-25T11:36:29.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f748dbae700 1 --2- 172.21.15.94:0/101375268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7488102cf0 0x7f74881a0130 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7480009fd0 tx=0x7f7480004e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:29.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.442+0000 7f747e7fc700 1 -- 172.21.15.94:0/101375268 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748001f030 con 0x7f7488102cf0 2024-04-25T11:36:29.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.443+0000 7f748f611700 1 -- 172.21.15.94:0/101375268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74881a0a60 con 0x7f7488102cf0 2024-04-25T11:36:29.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.443+0000 7f747e7fc700 1 -- 172.21.15.94:0/101375268 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7480007a80 con 0x7f7488102cf0 2024-04-25T11:36:29.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.443+0000 7f747e7fc700 1 -- 172.21.15.94:0/101375268 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480004810 con 0x7f7488102cf0 2024-04-25T11:36:29.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.443+0000 7f748f611700 1 -- 172.21.15.94:0/101375268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74881a0f90 con 0x7f7488102cf0 2024-04-25T11:36:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.444+0000 7f748f611700 1 -- 172.21.15.94:0/101375268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7488069730 con 0x7f7488102cf0 2024-04-25T11:36:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.445+0000 7f747e7fc700 1 -- 172.21.15.94:0/101375268 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f7480003a00 con 0x7f7488102cf0 2024-04-25T11:36:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.445+0000 7f747e7fc700 1 --2- 172.21.15.94:0/101375268 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f74740a3bb0 0x7f74740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.445+0000 7f747e7fc700 1 -- 172.21.15.94:0/101375268 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f748000f020 con 0x7f7488102cf0 2024-04-25T11:36:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.445+0000 7f748d3ad700 1 --2- 172.21.15.94:0/101375268 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f74740a3bb0 0x7f74740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:29.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.446+0000 7f748d3ad700 1 --2- 172.21.15.94:0/101375268 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f74740a3bb0 0x7f74740a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7484005fd0 tx=0x7f7484005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:29.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.450+0000 7f747e7fc700 1 -- 172.21.15.94:0/101375268 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7480029020 con 0x7f7488102cf0 2024-04-25T11:36:29.473 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:29 smithi111 ceph-mon[32784]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T11:36:29.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:29 smithi094 ceph-mon[33032]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T11:36:29.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:29.660+0000 7f748f611700 1 -- 172.21.15.94:0/101375268 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f74881a11c0 con 0x7f74740a3bb0 2024-04-25T11:36:29.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:29 smithi083 ceph-mon[29330]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T11:36:30.480 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:30 smithi094 ceph-mon[33032]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:36:30.480 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:30 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:36:30.480 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:30 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:36:30.480 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:30 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:30.480 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:30 smithi094 ceph-mon[33032]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:30.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:30 smithi111 ceph-mon[32784]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:36:30.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:30 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:36:30.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:30 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:36:30.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:30 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:30.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:30 smithi111 ceph-mon[32784]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:30.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:30 smithi083 ceph-mon[29330]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:36:30.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:36:30.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:36:30.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:30.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:30 smithi083 ceph-mon[29330]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:33.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:32 smithi083 ceph-mon[29330]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:33.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:32 smithi083 ceph-mon[29330]: from='client.? 172.21.15.94:0/1986722773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4540152-b87e-49a9-bf80-6de7cb165835"}]: dispatch 2024-04-25T11:36:33.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:32 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4540152-b87e-49a9-bf80-6de7cb165835"}]: dispatch 2024-04-25T11:36:33.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:32 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4540152-b87e-49a9-bf80-6de7cb165835"}]': finished 2024-04-25T11:36:33.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:32 smithi083 ceph-mon[29330]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T11:36:33.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:32 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:33.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:32 smithi094 ceph-mon[33032]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:33.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:32 smithi094 ceph-mon[33032]: from='client.? 172.21.15.94:0/1986722773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4540152-b87e-49a9-bf80-6de7cb165835"}]: dispatch 2024-04-25T11:36:33.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:32 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4540152-b87e-49a9-bf80-6de7cb165835"}]: dispatch 2024-04-25T11:36:33.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:32 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4540152-b87e-49a9-bf80-6de7cb165835"}]': finished 2024-04-25T11:36:33.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:32 smithi094 ceph-mon[33032]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T11:36:33.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:32 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:33.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:32 smithi111 ceph-mon[32784]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:33.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:32 smithi111 ceph-mon[32784]: from='client.? 172.21.15.94:0/1986722773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4540152-b87e-49a9-bf80-6de7cb165835"}]: dispatch 2024-04-25T11:36:33.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:32 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4540152-b87e-49a9-bf80-6de7cb165835"}]: dispatch 2024-04-25T11:36:33.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:32 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4540152-b87e-49a9-bf80-6de7cb165835"}]': finished 2024-04-25T11:36:33.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:32 smithi111 ceph-mon[32784]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T11:36:33.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:32 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:34.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:33 smithi083 ceph-mon[29330]: from='client.? 172.21.15.94:0/1003214056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:36:34.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:33 smithi094 ceph-mon[33032]: from='client.? 172.21.15.94:0/1003214056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:36:34.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:33 smithi111 ceph-mon[32784]: from='client.? 172.21.15.94:0/1003214056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:36:35.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:34 smithi083 ceph-mon[29330]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:35.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:34 smithi094 ceph-mon[33032]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:35.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:34 smithi111 ceph-mon[32784]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:37.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:36 smithi083 ceph-mon[29330]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:37.210 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:36 smithi094 ceph-mon[33032]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:37.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:36 smithi111 ceph-mon[32784]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:39.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:38 smithi083 ceph-mon[29330]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:39.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:38 smithi094 ceph-mon[33032]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:39.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:38 smithi111 ceph-mon[32784]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:40.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:39 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T11:36:40.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:39 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:40.165 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:39 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T11:36:40.165 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:39 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:40.166 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:39 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T11:36:40.166 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:39 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:41.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:40 smithi111 ceph-mon[32784]: Deploying daemon osd.1 on smithi094 2024-04-25T11:36:41.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:40 smithi111 ceph-mon[32784]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:41.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:40 smithi083 ceph-mon[29330]: Deploying daemon osd.1 on smithi094 2024-04-25T11:36:41.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:40 smithi083 ceph-mon[29330]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:41.212 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:40 smithi094 ceph-mon[33032]: Deploying daemon osd.1 on smithi094 2024-04-25T11:36:41.213 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:40 smithi094 ceph-mon[33032]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:43.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:42 smithi094 ceph-mon[33032]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:43.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:42 smithi111 ceph-mon[32784]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:43.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:42 smithi083 ceph-mon[29330]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:44 smithi094 ceph-mon[33032]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:44 smithi111 ceph-mon[32784]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:45.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:44 smithi083 ceph-mon[29330]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:46.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:46 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:46.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:46 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:46.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:46 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:46.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:46 smithi083 ceph-mon[29330]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:46.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:46 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:46.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:46 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:46.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:46 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:46.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:46 smithi094 ceph-mon[33032]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:46.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:46 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:46.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:46 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:46.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:46 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:46.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:46 smithi111 ceph-mon[32784]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:48.265 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 1 on host 'smithi094' 2024-04-25T11:36:48.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:48.276+0000 7f747e7fc700 1 -- 172.21.15.94:0/101375268 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f74881a11c0 con 0x7f74740a3bb0 2024-04-25T11:36:48.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:48.279+0000 7f748f611700 1 -- 172.21.15.94:0/101375268 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f74740a3bb0 msgr2=0x7f74740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:48.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:48.279+0000 7f748f611700 1 --2- 172.21.15.94:0/101375268 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f74740a3bb0 0x7f74740a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7484005fd0 tx=0x7f7484005e50 comp rx=0 tx=0).stop 2024-04-25T11:36:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:48.279+0000 7f748f611700 1 -- 172.21.15.94:0/101375268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7488102cf0 msgr2=0x7f74881a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:48.279+0000 7f748f611700 1 --2- 172.21.15.94:0/101375268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7488102cf0 0x7f74881a0130 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7480009fd0 tx=0x7f7480004e40 comp rx=0 tx=0).stop 2024-04-25T11:36:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:48.279+0000 7f748f611700 1 -- 172.21.15.94:0/101375268 shutdown_connections 2024-04-25T11:36:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:48.279+0000 7f748f611700 1 --2- 172.21.15.94:0/101375268 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f74740a3bb0 0x7f74740a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:48.279+0000 7f748f611700 1 --2- 172.21.15.94:0/101375268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7488102cf0 0x7f74881a0130 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:48.279+0000 7f748f611700 1 --2- 172.21.15.94:0/101375268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7488101150 0x7f7488198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:48.279+0000 7f748f611700 1 --2- 172.21.15.94:0/101375268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7488102350 0x7f7488199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:48.279+0000 7f748f611700 1 -- 172.21.15.94:0/101375268 >> 172.21.15.94:0/101375268 conn(0x7f74880fc6f0 msgr2=0x7f74880fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:48.279+0000 7f748f611700 1 -- 172.21.15.94:0/101375268 shutdown_connections 2024-04-25T11:36:48.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:36:48.279+0000 7f748f611700 1 -- 172.21.15.94:0/101375268 wait complete. 2024-04-25T11:36:48.871 DEBUG:teuthology.orchestra.run.smithi094:osd.1> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T11:36:48.874 INFO:tasks.cephadm:Deploying osd.2 on smithi111 with /dev/vg_nvme/lv_4... 2024-04-25T11:36:48.874 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T11:36:49.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:48 smithi094 ceph-mon[33032]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:49.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:48 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:49.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:48 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:49.024 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:36:48 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-1[37640]: 2024-04-25T11:36:48.973+0000 7faa347507c0 -1 osd.1 0 log_to_monitors true 2024-04-25T11:36:49.119 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:36:49.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:48 smithi083 ceph-mon[29330]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:49.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:48 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:49.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:48 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:49.240 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:48 smithi111 ceph-mon[32784]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:49.240 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:48 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:49.240 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:48 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:50.215 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:49 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:50.215 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:49 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:50.215 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:49 smithi094 ceph-mon[33032]: from='osd.1 [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:36:50.215 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:49 smithi094 ceph-mon[33032]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:36:50.248 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:49 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:50.248 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:49 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:50.248 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:49 smithi111 ceph-mon[32784]: from='osd.1 [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:36:50.248 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:49 smithi111 ceph-mon[32784]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:36:50.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:50.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:50.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:49 smithi083 ceph-mon[29330]: from='osd.1 [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:36:50.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:49 smithi083 ceph-mon[29330]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T11:36:50.866 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T11:36:50.867 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T11:36:50.867 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-25T11:36:50.867 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-25T11:36:50.867 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112298 s, 934 MB/s 2024-04-25T11:36:50.867 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-25T11:36:51.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:50 smithi111 ceph-mon[32784]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:51.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:50 smithi111 ceph-mon[32784]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T11:36:51.188 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:50 smithi111 ceph-mon[32784]: from='osd.1 [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:36:51.189 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:50 smithi111 ceph-mon[32784]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T11:36:51.189 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:51.189 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:50 smithi111 ceph-mon[32784]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:36:51.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:50 smithi094 ceph-mon[33032]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:51.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:50 smithi094 ceph-mon[33032]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T11:36:51.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:50 smithi094 ceph-mon[33032]: from='osd.1 [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:36:51.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:50 smithi094 ceph-mon[33032]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T11:36:51.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:51.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:50 smithi094 ceph-mon[33032]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:36:51.274 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:36:50 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-1[37640]: 2024-04-25T11:36:50.960+0000 7faa29db5700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T11:36:51.274 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:36:50 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-1[37640]: 2024-04-25T11:36:50.963+0000 7faa25ba8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:36:51.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:50 smithi083 ceph-mon[29330]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:51.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:50 smithi083 ceph-mon[29330]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T11:36:51.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:50 smithi083 ceph-mon[29330]: from='osd.1 [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:36:51.310 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:50 smithi083 ceph-mon[29330]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T11:36:51.310 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:51.310 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:50 smithi083 ceph-mon[29330]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:36:51.457 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_4 2024-04-25T11:36:51.697 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:36:52.066 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:51 smithi111 ceph-mon[32784]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T11:36:52.066 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:51 smithi111 ceph-mon[32784]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T11:36:52.066 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:51 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:52.067 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:51 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:52.215 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:51 smithi094 ceph-mon[33032]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T11:36:52.215 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:51 smithi094 ceph-mon[33032]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T11:36:52.215 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:51 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:52.215 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:51 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:52.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:51 smithi083 ceph-mon[29330]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T11:36:52.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:51 smithi083 ceph-mon[29330]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T11:36:52.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:51 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:52.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:51 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:53.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.251+0000 7f45efda4700 1 -- 172.21.15.111:0/3368092722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45e8103ca0 msgr2=0x7f45e8104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:53.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.251+0000 7f45efda4700 1 --2- 172.21.15.111:0/3368092722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45e8103ca0 0x7f45e8104100 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009990 tx=0x7f45d8009ca0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.251+0000 7f45efda4700 1 -- 172.21.15.111:0/3368092722 shutdown_connections 2024-04-25T11:36:53.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.251+0000 7f45efda4700 1 --2- 172.21.15.111:0/3368092722 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e8104640 0x7f45e810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.251+0000 7f45efda4700 1 --2- 172.21.15.111:0/3368092722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e8102aa0 0x7f45e8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.251+0000 7f45efda4700 1 --2- 172.21.15.111:0/3368092722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45e8103ca0 0x7f45e8104100 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.251+0000 7f45efda4700 1 -- 172.21.15.111:0/3368092722 >> 172.21.15.111:0/3368092722 conn(0x7f45e80fe040 msgr2=0x7f45e8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:36:53.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.251+0000 7f45efda4700 1 -- 172.21.15.111:0/3368092722 shutdown_connections 2024-04-25T11:36:53.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.251+0000 7f45efda4700 1 -- 172.21.15.111:0/3368092722 wait complete. 2024-04-25T11:36:53.242 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: purged_snaps scrub starts 2024-04-25T11:36:53.242 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: purged_snaps scrub ok 2024-04-25T11:36:53.242 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:53.242 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:53.243 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: osd.1 [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380] boot 2024-04-25T11:36:53.243 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T11:36:53.243 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:53.243 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:53.243 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:53.243 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:36:53.243 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:53.243 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:53.243 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:53.243 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:53.243 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:53.243 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:53.243 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:53.243 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:53.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.252+0000 7f45efda4700 1 Processor -- start 2024-04-25T11:36:53.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.252+0000 7f45efda4700 1 -- start start 2024-04-25T11:36:53.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.252+0000 7f45efda4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e8102aa0 0x7f45e8194060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:53.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.252+0000 7f45efda4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45e8103ca0 0x7f45e81945a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:53.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.252+0000 7f45efda4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e8104640 0x7f45e819b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:53.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.252+0000 7f45efda4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45e810e5b0 con 0x7f45e8104640 2024-04-25T11:36:53.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.252+0000 7f45efda4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f45e810e430 con 0x7f45e8102aa0 2024-04-25T11:36:53.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.252+0000 7f45efda4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f45e810e730 con 0x7f45e8103ca0 2024-04-25T11:36:53.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.252+0000 7f45ed33f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45e8103ca0 0x7f45e81945a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:53.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.252+0000 7f45ed33f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45e8103ca0 0x7f45e81945a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56396/0 (socket says 172.21.15.111:56396) 2024-04-25T11:36:53.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.252+0000 7f45ed33f700 1 -- 172.21.15.111:0/167284539 learned_addr learned my addr 172.21.15.111:0/167284539 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-25T11:36:53.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.253+0000 7f45ee341700 1 --2- 172.21.15.111:0/167284539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e8104640 0x7f45e819b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:53.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.253+0000 7f45edb40700 1 --2- 172.21.15.111:0/167284539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e8102aa0 0x7f45e8194060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:53.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.253+0000 7f45ed33f700 1 -- 172.21.15.111:0/167284539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e8102aa0 msgr2=0x7f45e8194060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:53.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.253+0000 7f45ed33f700 1 --2- 172.21.15.111:0/167284539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e8102aa0 0x7f45e8194060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.253+0000 7f45ed33f700 1 -- 172.21.15.111:0/167284539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e8104640 msgr2=0x7f45e819b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:36:53.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.253+0000 7f45ed33f700 1 --2- 172.21.15.111:0/167284539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e8104640 0x7f45e819b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:36:53.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.253+0000 7f45ed33f700 1 -- 172.21.15.111:0/167284539 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e819bcd0 con 0x7f45e8103ca0 2024-04-25T11:36:53.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.253+0000 7f45ee341700 1 --2- 172.21.15.111:0/167284539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e8104640 0x7f45e819b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:36:53.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.253+0000 7f45ed33f700 1 --2- 172.21.15.111:0/167284539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45e8103ca0 0x7f45e81945a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f45d80036f0 tx=0x7f45d80039f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:53.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.253+0000 7f45deffd700 1 -- 172.21.15.111:0/167284539 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8027070 con 0x7f45e8103ca0 2024-04-25T11:36:53.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.253+0000 7f45deffd700 1 -- 172.21.15.111:0/167284539 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45d8003f60 con 0x7f45e8103ca0 2024-04-25T11:36:53.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.253+0000 7f45efda4700 1 -- 172.21.15.111:0/167284539 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e819bf60 con 0x7f45e8103ca0 2024-04-25T11:36:53.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.253+0000 7f45deffd700 1 -- 172.21.15.111:0/167284539 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8022e60 con 0x7f45e8103ca0 2024-04-25T11:36:53.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.253+0000 7f45efda4700 1 -- 172.21.15.111:0/167284539 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e81afef0 con 0x7f45e8103ca0 2024-04-25T11:36:53.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.254+0000 7f45efda4700 1 -- 172.21.15.111:0/167284539 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e8069730 con 0x7f45e8103ca0 2024-04-25T11:36:53.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.256+0000 7f45deffd700 1 -- 172.21.15.111:0/167284539 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f45d802a070 con 0x7f45e8103ca0 2024-04-25T11:36:53.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.256+0000 7f45deffd700 1 --2- 172.21.15.111:0/167284539 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f45d40a3c30 0x7f45d40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:36:53.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.256+0000 7f45deffd700 1 -- 172.21.15.111:0/167284539 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f45d80d00a0 con 0x7f45e8103ca0 2024-04-25T11:36:53.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.256+0000 7f45edb40700 1 --2- 172.21.15.111:0/167284539 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f45d40a3c30 0x7f45d40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:36:53.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.257+0000 7f45edb40700 1 --2- 172.21.15.111:0/167284539 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f45d40a3c30 0x7f45d40a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f45e4006fd0 tx=0x7f45e4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:36:53.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.260+0000 7f45deffd700 1 -- 172.21.15.111:0/167284539 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45d8013070 con 0x7f45e8103ca0 2024-04-25T11:36:53.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: purged_snaps scrub starts 2024-04-25T11:36:53.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: purged_snaps scrub ok 2024-04-25T11:36:53.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:53.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:53.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: osd.1 [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380] boot 2024-04-25T11:36:53.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T11:36:53.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:53.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:53.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:53.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:36:53.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:53.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:53.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:53.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:53.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:53.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:53.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:53.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:53.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: purged_snaps scrub starts 2024-04-25T11:36:53.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: purged_snaps scrub ok 2024-04-25T11:36:53.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T11:36:53.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:53.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: osd.1 [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380] boot 2024-04-25T11:36:53.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T11:36:53.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:36:53.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:53.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:53.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:36:53.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:53.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:53.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:53.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:53.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:36:53.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:53.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:36:53.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:36:53.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:36:53.465+0000 7f45efda4700 1 -- 172.21.15.111:0/167284539 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f45e8062e80 con 0x7f45d40a3c30 2024-04-25T11:36:54.207 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:53 smithi111 ceph-mon[32784]: Detected new or changed devices on smithi094 2024-04-25T11:36:54.208 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:53 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi094 to 17088M 2024-04-25T11:36:54.208 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:53 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:36:54.208 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:53 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:36:54.208 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:53 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:54.208 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:53 smithi111 ceph-mon[32784]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T11:36:54.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:53 smithi094 ceph-mon[33032]: Detected new or changed devices on smithi094 2024-04-25T11:36:54.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:53 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi094 to 17088M 2024-04-25T11:36:54.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:53 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:36:54.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:53 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:36:54.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:53 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:54.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:53 smithi094 ceph-mon[33032]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T11:36:54.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:53 smithi083 ceph-mon[29330]: Detected new or changed devices on smithi094 2024-04-25T11:36:54.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:53 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi094 to 17088M 2024-04-25T11:36:54.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:53 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:36:54.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:53 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:36:54.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:53 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:36:54.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:53 smithi083 ceph-mon[29330]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T11:36:55.169 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:54 smithi111 ceph-mon[32784]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:36:55.169 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:54 smithi111 ceph-mon[32784]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:36:55.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:54 smithi094 ceph-mon[33032]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:36:55.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:54 smithi094 ceph-mon[33032]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:36:55.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:54 smithi083 ceph-mon[29330]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:36:55.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:54 smithi083 ceph-mon[29330]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:36:57.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:56 smithi094 ceph-mon[33032]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:36:57.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:56 smithi094 ceph-mon[33032]: from='client.? 172.21.15.111:0/4021020840' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdc6f455-7038-4178-b962-2de2d69180cb"}]: dispatch 2024-04-25T11:36:57.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:56 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdc6f455-7038-4178-b962-2de2d69180cb"}]: dispatch 2024-04-25T11:36:57.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:56 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bdc6f455-7038-4178-b962-2de2d69180cb"}]': finished 2024-04-25T11:36:57.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:56 smithi094 ceph-mon[33032]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T11:36:57.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:56 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:36:57.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:56 smithi094 ceph-mon[33032]: from='client.? 172.21.15.111:0/1465759406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:36:57.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:57 smithi111 ceph-mon[32784]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:36:57.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:57 smithi111 ceph-mon[32784]: from='client.? 172.21.15.111:0/4021020840' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdc6f455-7038-4178-b962-2de2d69180cb"}]: dispatch 2024-04-25T11:36:57.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:57 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdc6f455-7038-4178-b962-2de2d69180cb"}]: dispatch 2024-04-25T11:36:57.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:57 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bdc6f455-7038-4178-b962-2de2d69180cb"}]': finished 2024-04-25T11:36:57.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:57 smithi111 ceph-mon[32784]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T11:36:57.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:57 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:36:57.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:57 smithi111 ceph-mon[32784]: from='client.? 172.21.15.111:0/1465759406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:36:57.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:56 smithi083 ceph-mon[29330]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:36:57.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:56 smithi083 ceph-mon[29330]: from='client.? 172.21.15.111:0/4021020840' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdc6f455-7038-4178-b962-2de2d69180cb"}]: dispatch 2024-04-25T11:36:57.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:56 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdc6f455-7038-4178-b962-2de2d69180cb"}]: dispatch 2024-04-25T11:36:57.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:56 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bdc6f455-7038-4178-b962-2de2d69180cb"}]': finished 2024-04-25T11:36:57.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:56 smithi083 ceph-mon[29330]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T11:36:57.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:56 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:36:57.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:56 smithi083 ceph-mon[29330]: from='client.? 172.21.15.111:0/1465759406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:36:59.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:36:59 smithi094 ceph-mon[33032]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:36:59.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:36:59 smithi111 ceph-mon[32784]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:36:59.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:36:59 smithi083 ceph-mon[29330]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:01.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:01 smithi111 ceph-mon[32784]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:01.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:01 smithi083 ceph-mon[29330]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:01.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:01 smithi094 ceph-mon[33032]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:03.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:03 smithi094 ceph-mon[33032]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:03.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:03 smithi111 ceph-mon[32784]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:03.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:03 smithi083 ceph-mon[29330]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:04.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T11:37:04.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:04.286 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T11:37:04.287 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:04.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T11:37:04.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:05.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:05 smithi094 ceph-mon[33032]: Deploying daemon osd.2 on smithi111 2024-04-25T11:37:05.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:05 smithi094 ceph-mon[33032]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:05.309 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:05 smithi111 ceph-mon[32784]: Deploying daemon osd.2 on smithi111 2024-04-25T11:37:05.309 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:05 smithi111 ceph-mon[32784]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:05.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:05 smithi083 ceph-mon[29330]: Deploying daemon osd.2 on smithi111 2024-04-25T11:37:05.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:05 smithi083 ceph-mon[29330]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:07.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:07 smithi094 ceph-mon[33032]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:07.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:07 smithi111 ceph-mon[32784]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:07.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:07 smithi083 ceph-mon[29330]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:09.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:09 smithi094 ceph-mon[33032]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:09.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:09 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:09.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:09 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:09.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:09 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:09.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:09 smithi111 ceph-mon[32784]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:09.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:09 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:09.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:09 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:09.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:09 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:09.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:09 smithi083 ceph-mon[29330]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:09.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:09 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:09.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:09 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:09.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:09 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:11.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:11 smithi094 ceph-mon[33032]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:11.316 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:11 smithi111 ceph-mon[32784]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:11.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:11 smithi083 ceph-mon[29330]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:12.520 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:37:12.532+0000 7f45deffd700 1 -- 172.21.15.111:0/167284539 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f45e8062e80 con 0x7f45d40a3c30 2024-04-25T11:37:12.520 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 2 on host 'smithi111' 2024-04-25T11:37:12.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:37:12.534+0000 7f45efda4700 1 -- 172.21.15.111:0/167284539 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f45d40a3c30 msgr2=0x7f45d40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:12.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:37:12.534+0000 7f45efda4700 1 --2- 172.21.15.111:0/167284539 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f45d40a3c30 0x7f45d40a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f45e4006fd0 tx=0x7f45e4008040 comp rx=0 tx=0).stop 2024-04-25T11:37:12.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:37:12.534+0000 7f45efda4700 1 -- 172.21.15.111:0/167284539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45e8103ca0 msgr2=0x7f45e81945a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:12.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:37:12.534+0000 7f45efda4700 1 --2- 172.21.15.111:0/167284539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45e8103ca0 0x7f45e81945a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f45d80036f0 tx=0x7f45d80039f0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:37:12.534+0000 7f45efda4700 1 -- 172.21.15.111:0/167284539 shutdown_connections 2024-04-25T11:37:12.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:37:12.534+0000 7f45efda4700 1 --2- 172.21.15.111:0/167284539 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f45d40a3c30 0x7f45d40a60f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:37:12.534+0000 7f45efda4700 1 --2- 172.21.15.111:0/167284539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45e8104640 0x7f45e819b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:37:12.534+0000 7f45efda4700 1 --2- 172.21.15.111:0/167284539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45e8102aa0 0x7f45e8194060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:37:12.534+0000 7f45efda4700 1 --2- 172.21.15.111:0/167284539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45e8103ca0 0x7f45e81945a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:12.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:37:12.534+0000 7f45efda4700 1 -- 172.21.15.111:0/167284539 >> 172.21.15.111:0/167284539 conn(0x7f45e80fe040 msgr2=0x7f45e81050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:12.522 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:37:12.534+0000 7f45efda4700 1 -- 172.21.15.111:0/167284539 shutdown_connections 2024-04-25T11:37:12.523 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:37:12.534+0000 7f45efda4700 1 -- 172.21.15.111:0/167284539 wait complete. 2024-04-25T11:37:12.805 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:12.805 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:12.805 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:12.805 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:37:12.805 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:12.806 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:12 smithi111 ceph-mon[32784]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:12.806 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:12.806 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:12.806 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:12.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:12.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:12.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:12.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:37:12.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:12.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:12 smithi083 ceph-mon[29330]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:12.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:12.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:12.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:13.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:13.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:13.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:13.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:37:13.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:13.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:12 smithi094 ceph-mon[33032]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:13.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:13.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:13.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:13.114 DEBUG:teuthology.orchestra.run.smithi111:osd.2> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T11:37:13.117 INFO:tasks.cephadm:Deploying osd.3 on smithi083 with /dev/vg_nvme/lv_3... 2024-04-25T11:37:13.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T11:37:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:37:13.763 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:13 smithi083 ceph-mon[29330]: from='osd.2 [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T11:37:13.763 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:13 smithi083 ceph-mon[29330]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T11:37:13.978 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:13 smithi111 ceph-mon[32784]: from='osd.2 [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T11:37:13.978 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:13 smithi111 ceph-mon[32784]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T11:37:14.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:13 smithi094 ceph-mon[33032]: from='osd.2 [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T11:37:14.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:13 smithi094 ceph-mon[33032]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T11:37:14.784 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:14 smithi083 ceph-mon[29330]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T11:37:14.784 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:14 smithi083 ceph-mon[29330]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T11:37:14.784 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:14 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:37:14.784 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:14 smithi083 ceph-mon[29330]: from='osd.2 [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:37:14.784 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:14 smithi083 ceph-mon[29330]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:37:14.784 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:14 smithi083 ceph-mon[29330]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:14.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:14 smithi111 ceph-mon[32784]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T11:37:14.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:14 smithi111 ceph-mon[32784]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T11:37:14.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:14 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:37:14.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:14 smithi111 ceph-mon[32784]: from='osd.2 [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:37:14.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:14 smithi111 ceph-mon[32784]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:37:14.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:14 smithi111 ceph-mon[32784]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:14.823 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:37:14 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-2[37914]: 2024-04-25T11:37:14.665+0000 7ff7f74bc700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T11:37:14.823 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:37:14 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-2[37914]: 2024-04-25T11:37:14.669+0000 7ff7f32af700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:37:15.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:14 smithi094 ceph-mon[33032]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T11:37:15.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:14 smithi094 ceph-mon[33032]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T11:37:15.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:14 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:37:15.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:14 smithi094 ceph-mon[33032]: from='osd.2 [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:37:15.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:14 smithi094 ceph-mon[33032]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:37:15.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:14 smithi094 ceph-mon[33032]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:15.353 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T11:37:15.354 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T11:37:15.354 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-25T11:37:15.354 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-25T11:37:15.354 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114951 s, 912 MB/s 2024-04-25T11:37:15.354 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-25T11:37:15.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:15 smithi083 ceph-mon[29330]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-25T11:37:15.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:15 smithi083 ceph-mon[29330]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T11:37:15.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:15 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:37:15.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:15 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:37:15.967 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:15 smithi111 ceph-mon[32784]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-25T11:37:15.968 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:15 smithi111 ceph-mon[32784]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T11:37:15.968 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:15 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:37:15.968 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:15 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:37:16.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_3 2024-04-25T11:37:16.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:15 smithi094 ceph-mon[33032]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-25T11:37:16.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:15 smithi094 ceph-mon[33032]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T11:37:16.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:15 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:37:16.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:15 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:37:16.264 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:37:17.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:16 smithi094 ceph-mon[33032]: purged_snaps scrub starts 2024-04-25T11:37:17.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:16 smithi094 ceph-mon[33032]: purged_snaps scrub ok 2024-04-25T11:37:17.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:16 smithi094 ceph-mon[33032]: osd.2 [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764] boot 2024-04-25T11:37:17.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:16 smithi094 ceph-mon[33032]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T11:37:17.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:16 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:37:17.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:16 smithi094 ceph-mon[33032]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:17.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:16 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:37:17.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:16 smithi094 ceph-mon[33032]: Detected new or changed devices on smithi111 2024-04-25T11:37:17.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:16 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:17.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:16 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:17.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:16 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:37:17.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:16 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:17.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:16 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:17.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:16 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:37:17.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:16 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:17.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:16 smithi083 ceph-mon[29330]: purged_snaps scrub starts 2024-04-25T11:37:17.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:16 smithi083 ceph-mon[29330]: purged_snaps scrub ok 2024-04-25T11:37:17.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:16 smithi083 ceph-mon[29330]: osd.2 [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764] boot 2024-04-25T11:37:17.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:16 smithi083 ceph-mon[29330]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T11:37:17.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:16 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:37:17.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:16 smithi083 ceph-mon[29330]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:17.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:16 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:37:17.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:16 smithi083 ceph-mon[29330]: Detected new or changed devices on smithi111 2024-04-25T11:37:17.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:16 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:17.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:16 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:17.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:16 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:37:17.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:16 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:17.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:16 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:17.064 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:16 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:37:17.065 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:16 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:17.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:16 smithi111 ceph-mon[32784]: purged_snaps scrub starts 2024-04-25T11:37:17.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:16 smithi111 ceph-mon[32784]: purged_snaps scrub ok 2024-04-25T11:37:17.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:16 smithi111 ceph-mon[32784]: osd.2 [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764] boot 2024-04-25T11:37:17.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:16 smithi111 ceph-mon[32784]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T11:37:17.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:16 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:37:17.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:16 smithi111 ceph-mon[32784]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T11:37:17.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:16 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:37:17.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:16 smithi111 ceph-mon[32784]: Detected new or changed devices on smithi111 2024-04-25T11:37:17.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:16 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:17.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:16 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:17.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:16 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:37:17.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:16 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:17.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:16 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:17.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:16 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:37:17.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:16 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:17.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.897+0000 7fdc3258b700 1 -- 172.21.15.83:0/4185943797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c105e60 msgr2=0x7fdc2c108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:17.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.897+0000 7fdc3258b700 1 --2- 172.21.15.83:0/4185943797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c105e60 0x7fdc2c108250 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009990 tx=0x7fdc20009ca0 comp rx=0 tx=0).stop 2024-04-25T11:37:17.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.897+0000 7fdc3258b700 1 -- 172.21.15.83:0/4185943797 shutdown_connections 2024-04-25T11:37:17.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.897+0000 7fdc3258b700 1 --2- 172.21.15.83:0/4185943797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc2c108790 0x7fdc2c10ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:17.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.897+0000 7fdc3258b700 1 --2- 172.21.15.83:0/4185943797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc2c103530 0x7fdc2c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:17.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.897+0000 7fdc3258b700 1 --2- 172.21.15.83:0/4185943797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c105e60 0x7fdc2c108250 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.897+0000 7fdc3258b700 1 -- 172.21.15.83:0/4185943797 >> 172.21.15.83:0/4185943797 conn(0x7fdc2c0fcf40 msgr2=0x7fdc2c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.897+0000 7fdc3258b700 1 -- 172.21.15.83:0/4185943797 shutdown_connections 2024-04-25T11:37:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.897+0000 7fdc3258b700 1 -- 172.21.15.83:0/4185943797 wait complete. 2024-04-25T11:37:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.898+0000 7fdc3258b700 1 Processor -- start 2024-04-25T11:37:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.898+0000 7fdc3258b700 1 -- start start 2024-04-25T11:37:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.898+0000 7fdc3258b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc2c103530 0x7fdc2c102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.898+0000 7fdc3258b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c105e60 0x7fdc2c1014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.898+0000 7fdc3258b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc2c108790 0x7fdc2c101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.898+0000 7fdc3258b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c10e550 con 0x7fdc2c105e60 2024-04-25T11:37:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.898+0000 7fdc3258b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c10e3d0 con 0x7fdc2c108790 2024-04-25T11:37:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.898+0000 7fdc3258b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c10e6d0 con 0x7fdc2c103530 2024-04-25T11:37:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.898+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c105e60 0x7fdc2c1014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.898+0000 7fdc2b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c105e60 0x7fdc2c1014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58708/0 (socket says 172.21.15.83:58708) 2024-04-25T11:37:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.898+0000 7fdc2b7fe700 1 -- 172.21.15.83:0/334298348 learned_addr learned my addr 172.21.15.83:0/334298348 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:37:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.898+0000 7fdc2bfff700 1 --2- 172.21.15.83:0/334298348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc2c103530 0x7fdc2c102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.899+0000 7fdc30b28700 1 --2- 172.21.15.83:0/334298348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc2c108790 0x7fdc2c101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.899+0000 7fdc2bfff700 1 -- 172.21.15.83:0/334298348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc2c108790 msgr2=0x7fdc2c101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.899+0000 7fdc2bfff700 1 --2- 172.21.15.83:0/334298348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc2c108790 0x7fdc2c101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.899+0000 7fdc2bfff700 1 -- 172.21.15.83:0/334298348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c105e60 msgr2=0x7fdc2c1014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.899+0000 7fdc2bfff700 1 --2- 172.21.15.83:0/334298348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c105e60 0x7fdc2c1014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.899+0000 7fdc2bfff700 1 -- 172.21.15.83:0/334298348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c1022d0 con 0x7fdc2c103530 2024-04-25T11:37:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.899+0000 7fdc2bfff700 1 --2- 172.21.15.83:0/334298348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc2c103530 0x7fdc2c102e90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00e9f0 tx=0x7fdc1c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.899+0000 7fdc297fa700 1 -- 172.21.15.83:0/334298348 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c004220 con 0x7fdc2c103530 2024-04-25T11:37:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.899+0000 7fdc3258b700 1 -- 172.21.15.83:0/334298348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2c1a2dc0 con 0x7fdc2c103530 2024-04-25T11:37:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.899+0000 7fdc297fa700 1 -- 172.21.15.83:0/334298348 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc1c0043c0 con 0x7fdc2c103530 2024-04-25T11:37:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.899+0000 7fdc297fa700 1 -- 172.21.15.83:0/334298348 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c005020 con 0x7fdc2c103530 2024-04-25T11:37:17.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.899+0000 7fdc3258b700 1 -- 172.21.15.83:0/334298348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc2c1a3160 con 0x7fdc2c103530 2024-04-25T11:37:17.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.900+0000 7fdc3258b700 1 -- 172.21.15.83:0/334298348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc2c069730 con 0x7fdc2c103530 2024-04-25T11:37:17.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.902+0000 7fdc297fa700 1 -- 172.21.15.83:0/334298348 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fdc1c004750 con 0x7fdc2c103530 2024-04-25T11:37:17.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.902+0000 7fdc297fa700 1 --2- 172.21.15.83:0/334298348 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdc140a3bb0 0x7fdc140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:17.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.903+0000 7fdc2b7fe700 1 --2- 172.21.15.83:0/334298348 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdc140a3bb0 0x7fdc140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:17.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.903+0000 7fdc297fa700 1 -- 172.21.15.83:0/334298348 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7fdc1c0cee60 con 0x7fdc2c103530 2024-04-25T11:37:17.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.903+0000 7fdc2b7fe700 1 --2- 172.21.15.83:0/334298348 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdc140a3bb0 0x7fdc140a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000b2e0 tx=0x7fdc20005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:17.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:17 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi111 to 17088M 2024-04-25T11:37:17.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:17 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T11:37:17.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:17 smithi083 ceph-mon[29330]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T11:37:17.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:17 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:37:17.891 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:37:17 smithi083 sudo[48731]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T11:37:17.892 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:37:17 smithi083 sudo[48731]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:37:17.892 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:37:17 smithi083 sudo[48731]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:37:17.892 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:37:17 smithi083 sudo[48731]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:37:17.892 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:37:17 smithi083 sudo[48735]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T11:37:17.892 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:37:17 smithi083 sudo[48735]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:37:17.892 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:37:17 smithi083 sudo[48735]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:37:17.892 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:37:17 smithi083 sudo[48735]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:37:17.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:17.908+0000 7fdc297fa700 1 -- 172.21.15.83:0/334298348 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc1c09ccf0 con 0x7fdc2c103530 2024-04-25T11:37:18.006 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:17 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi111 to 17088M 2024-04-25T11:37:18.006 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:17 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T11:37:18.006 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:17 smithi111 ceph-mon[32784]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T11:37:18.006 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:17 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:37:18.007 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:37:17 smithi111 sudo[41962]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T11:37:18.007 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:37:17 smithi111 sudo[41962]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:37:18.007 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:37:17 smithi111 sudo[41962]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:37:18.007 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:37:17 smithi111 sudo[41962]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:37:18.007 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:37:17 smithi111 sudo[41965]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T11:37:18.007 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:37:17 smithi111 sudo[41965]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:37:18.007 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:37:17 smithi111 sudo[41965]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:37:18.007 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:37:17 smithi111 sudo[41965]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:37:18.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:17 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi111 to 17088M 2024-04-25T11:37:18.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:17 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T11:37:18.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:17 smithi094 ceph-mon[33032]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T11:37:18.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:17 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:37:18.023 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:37:17 smithi094 sudo[41322]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T11:37:18.023 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:37:17 smithi094 sudo[41322]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:37:18.024 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:37:17 smithi094 sudo[41322]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:37:18.024 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:37:17 smithi094 sudo[41322]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:37:18.024 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:37:17 smithi094 sudo[41325]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T11:37:18.024 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:37:17 smithi094 sudo[41325]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:37:18.024 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:37:17 smithi094 sudo[41325]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:37:18.024 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:37:17 smithi094 sudo[41325]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:37:18.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:18.124+0000 7fdc3258b700 1 -- 172.21.15.83:0/334298348 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdc2c062e80 con 0x7fdc140a3bb0 2024-04-25T11:37:18.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:17 smithi083 sudo[48758]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T11:37:18.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:17 smithi083 sudo[48758]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:37:18.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:17 smithi083 sudo[48758]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:37:18.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:17 smithi083 sudo[48758]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:37:18.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 sudo[48761]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T11:37:18.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 sudo[48761]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:37:18.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 sudo[48761]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:37:18.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 sudo[48761]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:37:18.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 sudo[41968]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T11:37:18.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 sudo[41968]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:37:18.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 sudo[41968]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:37:18.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 sudo[41968]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:37:18.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 sudo[41971]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T11:37:18.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 sudo[41971]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:37:18.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 sudo[41971]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:37:18.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 sudo[41971]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:37:18.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 sudo[41328]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T11:37:18.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 sudo[41328]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:37:18.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 sudo[41328]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:37:18.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 sudo[41328]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:37:18.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 sudo[41331]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T11:37:18.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 sudo[41331]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T11:37:18.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 sudo[41331]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T11:37:18.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 sudo[41331]: pam_unix(sudo:session): session closed for user root 2024-04-25T11:37:18.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:18.859+0000 7fdc297fa700 1 -- 172.21.15.83:0/334298348 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdc1c004a40 con 0x7fdc2c103530 2024-04-25T11:37:18.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T11:37:18.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T11:37:18.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:18.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:18.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:18.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:18.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:37:18.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:37:18.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:37:18.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:37:18.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:37:18.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:37:18.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:37:18.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:37:18.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:37:18.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:37:18.964 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:37:18.964 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:37:18.964 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:37:18.964 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:37:18.964 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:37:18.964 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:37:18.964 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:37:18.964 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:18.964 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:18 smithi083 ceph-mon[29330]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T11:37:19.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T11:37:19.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T11:37:19.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:19.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:19.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:19.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:19.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:37:19.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:37:19.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:37:19.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:37:19.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:37:19.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:37:19.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:37:19.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:37:19.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:37:19.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:37:19.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:37:19.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:37:19.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:37:19.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:37:19.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:37:19.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:37:19.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:37:19.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:19.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:18 smithi094 ceph-mon[33032]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T11:37:19.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T11:37:19.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T11:37:19.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:19.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:19.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:19.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:19.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:37:19.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:37:19.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:37:19.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:37:19.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:37:19.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:37:19.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:37:19.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:37:19.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:37:19.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:37:19.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T11:37:19.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:37:19.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:37:19.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:37:19.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T11:37:19.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:37:19.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:37:19.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:19.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:18 smithi111 ceph-mon[32784]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T11:37:20.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:19 smithi083 ceph-mon[29330]: from='client.24181 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:37:20.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:19 smithi083 ceph-mon[29330]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T11:37:20.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:19 smithi094 ceph-mon[33032]: from='client.24181 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:37:20.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:19 smithi094 ceph-mon[33032]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T11:37:20.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:19 smithi111 ceph-mon[32784]: from='client.24181 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:37:20.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:19 smithi111 ceph-mon[32784]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T11:37:21.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:20 smithi083 ceph-mon[29330]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:21.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:20 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/4119768452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e927e50-a208-48ca-aeb4-1adc2ae48d55"}]: dispatch 2024-04-25T11:37:21.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:20 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/4119768452' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e927e50-a208-48ca-aeb4-1adc2ae48d55"}]': finished 2024-04-25T11:37:21.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:20 smithi083 ceph-mon[29330]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T11:37:21.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:21.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:20 smithi094 ceph-mon[33032]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:21.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:20 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/4119768452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e927e50-a208-48ca-aeb4-1adc2ae48d55"}]: dispatch 2024-04-25T11:37:21.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:20 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/4119768452' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e927e50-a208-48ca-aeb4-1adc2ae48d55"}]': finished 2024-04-25T11:37:21.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:20 smithi094 ceph-mon[33032]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T11:37:21.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:21.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:20 smithi111 ceph-mon[32784]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:21.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:20 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/4119768452' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e927e50-a208-48ca-aeb4-1adc2ae48d55"}]: dispatch 2024-04-25T11:37:21.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:20 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/4119768452' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e927e50-a208-48ca-aeb4-1adc2ae48d55"}]': finished 2024-04-25T11:37:21.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:20 smithi111 ceph-mon[32784]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T11:37:21.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:22.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:21 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1186497831' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:37:22.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:21 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1186497831' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:37:22.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:21 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1186497831' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:37:23.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:22 smithi083 ceph-mon[29330]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:23.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:22 smithi094 ceph-mon[33032]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:23.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:22 smithi111 ceph-mon[32784]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:25.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:24 smithi083 ceph-mon[29330]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 100 KiB/s wr, 6 op/s 2024-04-25T11:37:25.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:24 smithi094 ceph-mon[33032]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 100 KiB/s wr, 6 op/s 2024-04-25T11:37:25.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:24 smithi111 ceph-mon[32784]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 100 KiB/s wr, 6 op/s 2024-04-25T11:37:27.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:26 smithi083 ceph-mon[29330]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 72 KiB/s wr, 4 op/s 2024-04-25T11:37:27.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:26 smithi094 ceph-mon[33032]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 72 KiB/s wr, 4 op/s 2024-04-25T11:37:27.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:26 smithi111 ceph-mon[32784]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 72 KiB/s wr, 4 op/s 2024-04-25T11:37:29.108 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:28 smithi083 ceph-mon[29330]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 57 KiB/s wr, 3 op/s 2024-04-25T11:37:29.108 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:28 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T11:37:29.108 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:28 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:29.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:28 smithi094 ceph-mon[33032]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 57 KiB/s wr, 3 op/s 2024-04-25T11:37:29.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:28 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T11:37:29.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:28 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:29.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:28 smithi111 ceph-mon[32784]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.3 KiB/s rd, 57 KiB/s wr, 3 op/s 2024-04-25T11:37:29.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:28 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T11:37:29.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:28 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:30.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:29 smithi083 ceph-mon[29330]: Deploying daemon osd.3 on smithi083 2024-04-25T11:37:30.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:29 smithi094 ceph-mon[33032]: Deploying daemon osd.3 on smithi083 2024-04-25T11:37:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:29 smithi111 ceph-mon[32784]: Deploying daemon osd.3 on smithi083 2024-04-25T11:37:31.027 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:30 smithi083 ceph-mon[29330]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 52 KiB/s wr, 3 op/s 2024-04-25T11:37:31.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:30 smithi094 ceph-mon[33032]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 52 KiB/s wr, 3 op/s 2024-04-25T11:37:31.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:30 smithi111 ceph-mon[32784]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 52 KiB/s wr, 3 op/s 2024-04-25T11:37:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:32 smithi083 ceph-mon[29330]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 47 KiB/s wr, 3 op/s 2024-04-25T11:37:33.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:32 smithi094 ceph-mon[33032]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 47 KiB/s wr, 3 op/s 2024-04-25T11:37:33.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:32 smithi111 ceph-mon[32784]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 47 KiB/s wr, 3 op/s 2024-04-25T11:37:34.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:34.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:34.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:34.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:34 smithi083 ceph-mon[29330]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 43 KiB/s wr, 2 op/s 2024-04-25T11:37:35.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:35.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:35.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:35.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:34 smithi094 ceph-mon[33032]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 43 KiB/s wr, 2 op/s 2024-04-25T11:37:35.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:35.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:35.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:35.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:34 smithi111 ceph-mon[32784]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 43 KiB/s wr, 2 op/s 2024-04-25T11:37:37.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:36 smithi094 ceph-mon[33032]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:37.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:36 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:37.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:36 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:37.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:36 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:37.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:36 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:37:37.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:36 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:37.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:36 smithi111 ceph-mon[32784]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:37.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:36 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:37.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:36 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:37.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:36 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:37.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:36 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:37:37.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:36 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:37.339 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:36 smithi083 ceph-mon[29330]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:37.339 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:36 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:37.339 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:36 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:37.339 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:36 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:37.339 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:36 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:37:37.339 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:36 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:37.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:37.435+0000 7fdc297fa700 1 -- 172.21.15.83:0/334298348 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdc2c062e80 con 0x7fdc140a3bb0 2024-04-25T11:37:37.423 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 3 on host 'smithi083' 2024-04-25T11:37:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:37.437+0000 7fdc3258b700 1 -- 172.21.15.83:0/334298348 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdc140a3bb0 msgr2=0x7fdc140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:37.437+0000 7fdc3258b700 1 --2- 172.21.15.83:0/334298348 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdc140a3bb0 0x7fdc140a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000b2e0 tx=0x7fdc20005fb0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:37.437+0000 7fdc3258b700 1 -- 172.21.15.83:0/334298348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc2c103530 msgr2=0x7fdc2c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:37.437+0000 7fdc3258b700 1 --2- 172.21.15.83:0/334298348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc2c103530 0x7fdc2c102e90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00e9f0 tx=0x7fdc1c00ed00 comp rx=0 tx=0).stop 2024-04-25T11:37:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:37.437+0000 7fdc3258b700 1 -- 172.21.15.83:0/334298348 shutdown_connections 2024-04-25T11:37:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:37.437+0000 7fdc3258b700 1 --2- 172.21.15.83:0/334298348 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdc140a3bb0 0x7fdc140a6070 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:37.437+0000 7fdc3258b700 1 --2- 172.21.15.83:0/334298348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc2c108790 0x7fdc2c101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:37.437+0000 7fdc3258b700 1 --2- 172.21.15.83:0/334298348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc2c103530 0x7fdc2c102e90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:37.437+0000 7fdc3258b700 1 --2- 172.21.15.83:0/334298348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc2c105e60 0x7fdc2c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:37.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:37.437+0000 7fdc3258b700 1 -- 172.21.15.83:0/334298348 >> 172.21.15.83:0/334298348 conn(0x7fdc2c0fcf40 msgr2=0x7fdc2c1053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:37.437+0000 7fdc3258b700 1 -- 172.21.15.83:0/334298348 shutdown_connections 2024-04-25T11:37:37.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:37:37.437+0000 7fdc3258b700 1 -- 172.21.15.83:0/334298348 wait complete. 2024-04-25T11:37:38.100 DEBUG:teuthology.orchestra.run.smithi083:osd.3> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T11:37:38.103 INFO:tasks.cephadm:Deploying osd.4 on smithi094 with /dev/vg_nvme/lv_3... 2024-04-25T11:37:38.103 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T11:37:38.349 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:37:38.464 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:38 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:38.465 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:38 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:38.465 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:38 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:38.465 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:38 smithi094 ceph-mon[33032]: from='osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T11:37:38.465 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:38 smithi094 ceph-mon[33032]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:38.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:38 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:38.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:38 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:38.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:38 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:38.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:38 smithi083 ceph-mon[29330]: from='osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T11:37:38.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:38 smithi083 ceph-mon[29330]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:38.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:38 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:38.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:38 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:38.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:38 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:38.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:38 smithi111 ceph-mon[32784]: from='osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T11:37:38.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:38 smithi111 ceph-mon[32784]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:39.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:39 smithi083 ceph-mon[29330]: from='osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T11:37:39.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:39 smithi083 ceph-mon[29330]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T11:37:39.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:39 smithi083 ceph-mon[29330]: from='osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T11:37:39.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:39 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:39.698 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:39 smithi094 ceph-mon[33032]: from='osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T11:37:39.698 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:39 smithi094 ceph-mon[33032]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T11:37:39.698 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:39 smithi094 ceph-mon[33032]: from='osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T11:37:39.698 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:39 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:39.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:39 smithi111 ceph-mon[32784]: from='osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T11:37:39.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:39 smithi111 ceph-mon[32784]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T11:37:39.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:39 smithi111 ceph-mon[32784]: from='osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T11:37:39.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:39 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:40.290 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T11:37:40.290 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T11:37:40.290 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-25T11:37:40.291 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-25T11:37:40.291 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0102602 s, 1.0 GB/s 2024-04-25T11:37:40.291 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-25T11:37:40.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:40 smithi094 ceph-mon[33032]: from='osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T11:37:40.609 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:40 smithi094 ceph-mon[33032]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T11:37:40.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:40 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:40.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:40 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:40.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:40 smithi094 ceph-mon[33032]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:40.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:40 smithi094 ceph-mon[33032]: from='osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209]' entity='osd.3' 2024-04-25T11:37:40.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:40 smithi094 ceph-mon[33032]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T11:37:40.610 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:40 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:40.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:40 smithi083 ceph-mon[29330]: from='osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T11:37:40.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:40 smithi083 ceph-mon[29330]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T11:37:40.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:40.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:40.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:40 smithi083 ceph-mon[29330]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:40.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:40 smithi083 ceph-mon[29330]: from='osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209]' entity='osd.3' 2024-04-25T11:37:40.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:40 smithi083 ceph-mon[29330]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T11:37:40.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:40.657 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:37:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-3[51025]: 2024-04-25T11:37:40.294+0000 7fbb78846700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T11:37:40.657 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:37:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-3[51025]: 2024-04-25T11:37:40.406+0000 7fbb6fe30700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:37:40.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:40 smithi111 ceph-mon[32784]: from='osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T11:37:40.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:40 smithi111 ceph-mon[32784]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T11:37:40.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:40.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:40.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:40 smithi111 ceph-mon[32784]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T11:37:40.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:40 smithi111 ceph-mon[32784]: from='osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209]' entity='osd.3' 2024-04-25T11:37:40.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:40 smithi111 ceph-mon[32784]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T11:37:40.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:40.929 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_3 2024-04-25T11:37:41.175 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:37:41.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:41 smithi094 ceph-mon[33032]: purged_snaps scrub starts 2024-04-25T11:37:41.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:41 smithi094 ceph-mon[33032]: purged_snaps scrub ok 2024-04-25T11:37:41.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:41 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:41.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:41 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:41.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:41 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:37:41.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:41 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:37:41.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:41 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:41.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:41 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:41.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:41 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:37:41.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:41 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:41.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:41 smithi094 ceph-mon[33032]: osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209] boot 2024-04-25T11:37:41.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:41 smithi094 ceph-mon[33032]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T11:37:41.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:41 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:41.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:41 smithi083 ceph-mon[29330]: purged_snaps scrub starts 2024-04-25T11:37:41.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:41 smithi083 ceph-mon[29330]: purged_snaps scrub ok 2024-04-25T11:37:41.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:41 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:41.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:41 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:41.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:41 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:37:41.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:41 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:37:41.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:41 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:41.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:41 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:41.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:41 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:37:41.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:41 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:41.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:41 smithi083 ceph-mon[29330]: osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209] boot 2024-04-25T11:37:41.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:41 smithi083 ceph-mon[29330]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T11:37:41.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:41 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:41.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:41 smithi111 ceph-mon[32784]: purged_snaps scrub starts 2024-04-25T11:37:41.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:41 smithi111 ceph-mon[32784]: purged_snaps scrub ok 2024-04-25T11:37:41.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:41 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:41.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:41 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:41.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:41 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:37:41.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:41 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:37:41.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:41 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:41.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:41 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:41.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:41 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:37:41.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:41 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:41.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:41 smithi111 ceph-mon[32784]: osd.3 [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209] boot 2024-04-25T11:37:41.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:41 smithi111 ceph-mon[32784]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T11:37:41.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:41 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:37:42.560 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:42 smithi094 ceph-mon[33032]: Detected new or changed devices on smithi083 2024-04-25T11:37:42.560 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:42 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-25T11:37:42.560 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:42 smithi094 ceph-mon[33032]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T11:37:42.560 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:42 smithi094 ceph-mon[33032]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:42.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.809+0000 7faee209b700 1 -- 172.21.15.94:0/1937018027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc105ef0 msgr2=0x7faedc1082e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:42.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.809+0000 7faee209b700 1 --2- 172.21.15.94:0/1937018027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc105ef0 0x7faedc1082e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faed0009990 tx=0x7faed0009ca0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.809+0000 7faee209b700 1 -- 172.21.15.94:0/1937018027 shutdown_connections 2024-04-25T11:37:42.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.809+0000 7faee209b700 1 --2- 172.21.15.94:0/1937018027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faedc108820 0x7faedc10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.809+0000 7faee209b700 1 --2- 172.21.15.94:0/1937018027 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faedc1035c0 0x7faedc1059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.809+0000 7faee209b700 1 --2- 172.21.15.94:0/1937018027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc105ef0 0x7faedc1082e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.809+0000 7faee209b700 1 -- 172.21.15.94:0/1937018027 >> 172.21.15.94:0/1937018027 conn(0x7faedc0fcf70 msgr2=0x7faedc0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:37:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.809+0000 7faee209b700 1 -- 172.21.15.94:0/1937018027 shutdown_connections 2024-04-25T11:37:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.809+0000 7faee209b700 1 -- 172.21.15.94:0/1937018027 wait complete. 2024-04-25T11:37:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.809+0000 7faee209b700 1 Processor -- start 2024-04-25T11:37:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faee209b700 1 -- start start 2024-04-25T11:37:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faee209b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faedc1035c0 0x7faedc198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faee209b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faedc105ef0 0x7faedc199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faee209b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc108820 0x7faedc1a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faee209b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faedc10e600 con 0x7faedc1035c0 2024-04-25T11:37:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faee209b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faedc10e480 con 0x7faedc108820 2024-04-25T11:37:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faee209b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faedc10e780 con 0x7faedc105ef0 2024-04-25T11:37:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faedbfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc108820 0x7faedc1a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faedbfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc108820 0x7faedc1a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53208/0 (socket says 172.21.15.94:53208) 2024-04-25T11:37:42.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faedbfff700 1 -- 172.21.15.94:0/2265257544 learned_addr learned my addr 172.21.15.94:0/2265257544 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T11:37:42.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faedb7fe700 1 --2- 172.21.15.94:0/2265257544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faedc1035c0 0x7faedc198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:42.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faedaffd700 1 --2- 172.21.15.94:0/2265257544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faedc105ef0 0x7faedc199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:42.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faedbfff700 1 -- 172.21.15.94:0/2265257544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faedc105ef0 msgr2=0x7faedc199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:42.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faedbfff700 1 --2- 172.21.15.94:0/2265257544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faedc105ef0 0x7faedc199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faedbfff700 1 -- 172.21.15.94:0/2265257544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faedc1035c0 msgr2=0x7faedc198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:37:42.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faedbfff700 1 --2- 172.21.15.94:0/2265257544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faedc1035c0 0x7faedc198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:37:42.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.810+0000 7faedbfff700 1 -- 172.21.15.94:0/2265257544 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faedc1a0950 con 0x7faedc108820 2024-04-25T11:37:42.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.811+0000 7faedbfff700 1 --2- 172.21.15.94:0/2265257544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc108820 0x7faedc1a0280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faed400b9e0 tx=0x7faed400bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:42.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.811+0000 7faed8ff9700 1 -- 172.21.15.94:0/2265257544 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4019070 con 0x7faedc108820 2024-04-25T11:37:42.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.811+0000 7faed8ff9700 1 -- 172.21.15.94:0/2265257544 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed400cde0 con 0x7faedc108820 2024-04-25T11:37:42.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.811+0000 7faee209b700 1 -- 172.21.15.94:0/2265257544 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faedc1a0c40 con 0x7faedc108820 2024-04-25T11:37:42.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.811+0000 7faed8ff9700 1 -- 172.21.15.94:0/2265257544 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4012340 con 0x7faedc108820 2024-04-25T11:37:42.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.811+0000 7faee209b700 1 -- 172.21.15.94:0/2265257544 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faedc1a1130 con 0x7faedc108820 2024-04-25T11:37:42.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.812+0000 7faee209b700 1 -- 172.21.15.94:0/2265257544 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faedc069730 con 0x7faedc108820 2024-04-25T11:37:42.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.813+0000 7faed8ff9700 1 -- 172.21.15.94:0/2265257544 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faed401c070 con 0x7faedc108820 2024-04-25T11:37:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.813+0000 7faed8ff9700 1 --2- 172.21.15.94:0/2265257544 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faec40a3c40 0x7faec40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:37:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.814+0000 7faed8ff9700 1 -- 172.21.15.94:0/2265257544 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7faed40ce670 con 0x7faedc108820 2024-04-25T11:37:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.814+0000 7faedb7fe700 1 --2- 172.21.15.94:0/2265257544 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faec40a3c40 0x7faec40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:37:42.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.814+0000 7faedb7fe700 1 --2- 172.21.15.94:0/2265257544 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faec40a3c40 0x7faec40a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faecc005950 tx=0x7faecc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:37:42.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:42.818+0000 7faed8ff9700 1 -- 172.21.15.94:0/2265257544 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faed409c250 con 0x7faedc108820 2024-04-25T11:37:42.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:42 smithi111 ceph-mon[32784]: Detected new or changed devices on smithi083 2024-04-25T11:37:42.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:42 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-25T11:37:42.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:42 smithi111 ceph-mon[32784]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T11:37:42.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:42 smithi111 ceph-mon[32784]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:42.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:42 smithi083 ceph-mon[29330]: Detected new or changed devices on smithi083 2024-04-25T11:37:42.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:42 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-25T11:37:42.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:42 smithi083 ceph-mon[29330]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T11:37:42.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:42 smithi083 ceph-mon[29330]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:43.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:37:43.022+0000 7faee209b700 1 -- 172.21.15.94:0/2265257544 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7faedc062e80 con 0x7faec40a3c40 2024-04-25T11:37:43.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:43 smithi111 ceph-mon[32784]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T11:37:43.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:43 smithi111 ceph-mon[32784]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:37:43.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:43 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:37:43.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:43 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:37:43.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:43 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:43.823 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:43 smithi094 ceph-mon[33032]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T11:37:43.823 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:43 smithi094 ceph-mon[33032]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:37:43.823 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:43 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:37:43.823 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:43 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:37:43.823 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:43 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:43.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:43 smithi083 ceph-mon[29330]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T11:37:43.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:43 smithi083 ceph-mon[29330]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:37:43.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:43 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:37:43.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:43 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:37:43.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:43 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:44.812 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:44 smithi094 ceph-mon[33032]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T11:37:44.812 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:44 smithi094 ceph-mon[33032]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 167 KiB/s, 0 objects/s recovering 2024-04-25T11:37:44.812 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:44 smithi094 ceph-mon[33032]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T11:37:44.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:44 smithi111 ceph-mon[32784]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T11:37:44.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:44 smithi111 ceph-mon[32784]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 167 KiB/s, 0 objects/s recovering 2024-04-25T11:37:44.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:44 smithi111 ceph-mon[32784]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T11:37:44.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:44 smithi083 ceph-mon[29330]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T11:37:44.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:44 smithi083 ceph-mon[29330]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 167 KiB/s, 0 objects/s recovering 2024-04-25T11:37:44.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:44 smithi083 ceph-mon[29330]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T11:37:45.995 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:45 smithi094 ceph-mon[33032]: from='client.? 172.21.15.94:0/7638897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "637e5b1b-12cb-41e9-9026-53faadfc28b6"}]: dispatch 2024-04-25T11:37:45.995 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:45 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "637e5b1b-12cb-41e9-9026-53faadfc28b6"}]: dispatch 2024-04-25T11:37:45.995 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:45 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "637e5b1b-12cb-41e9-9026-53faadfc28b6"}]': finished 2024-04-25T11:37:45.995 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:45 smithi094 ceph-mon[33032]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T11:37:45.995 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:45 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:37:46.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:45 smithi111 ceph-mon[32784]: from='client.? 172.21.15.94:0/7638897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "637e5b1b-12cb-41e9-9026-53faadfc28b6"}]: dispatch 2024-04-25T11:37:46.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:45 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "637e5b1b-12cb-41e9-9026-53faadfc28b6"}]: dispatch 2024-04-25T11:37:46.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:45 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "637e5b1b-12cb-41e9-9026-53faadfc28b6"}]': finished 2024-04-25T11:37:46.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:45 smithi111 ceph-mon[32784]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T11:37:46.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:45 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:37:46.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:45 smithi083 ceph-mon[29330]: from='client.? 172.21.15.94:0/7638897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "637e5b1b-12cb-41e9-9026-53faadfc28b6"}]: dispatch 2024-04-25T11:37:46.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:45 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "637e5b1b-12cb-41e9-9026-53faadfc28b6"}]: dispatch 2024-04-25T11:37:46.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:45 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "637e5b1b-12cb-41e9-9026-53faadfc28b6"}]': finished 2024-04-25T11:37:46.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:45 smithi083 ceph-mon[29330]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T11:37:46.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:45 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:37:47.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:46 smithi094 ceph-mon[33032]: pgmap v109: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T11:37:47.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:46 smithi094 ceph-mon[33032]: from='client.? 172.21.15.94:0/529168204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:37:47.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:46 smithi111 ceph-mon[32784]: pgmap v109: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T11:37:47.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:46 smithi111 ceph-mon[32784]: from='client.? 172.21.15.94:0/529168204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:37:47.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:46 smithi083 ceph-mon[29330]: pgmap v109: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T11:37:47.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:46 smithi083 ceph-mon[29330]: from='client.? 172.21.15.94:0/529168204' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:37:49.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:48 smithi083 ceph-mon[29330]: pgmap v110: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-25T11:37:49.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:48 smithi094 ceph-mon[33032]: pgmap v110: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-25T11:37:49.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:48 smithi111 ceph-mon[32784]: pgmap v110: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 113 MiB used, 357 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-25T11:37:50.156 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:37:49 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:37:49.842+0000 7f964f9ce700 -1 calc_pg_upmaps abort due to max <= 0 2024-04-25T11:37:51.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:50 smithi094 ceph-mon[33032]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:51.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:50 smithi083 ceph-mon[29330]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:51.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:50 smithi111 ceph-mon[32784]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:52 smithi083 ceph-mon[29330]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T11:37:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:53.251 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:52 smithi094 ceph-mon[33032]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:53.251 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T11:37:53.251 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:53.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:52 smithi111 ceph-mon[32784]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:53.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T11:37:53.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:37:54.157 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:53 smithi094 ceph-mon[33032]: Deploying daemon osd.4 on smithi094 2024-04-25T11:37:54.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:53 smithi111 ceph-mon[32784]: Deploying daemon osd.4 on smithi094 2024-04-25T11:37:54.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:53 smithi083 ceph-mon[29330]: Deploying daemon osd.4 on smithi094 2024-04-25T11:37:55.155 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:54 smithi094 ceph-mon[33032]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:55.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:54 smithi111 ceph-mon[32784]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:55.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:54 smithi083 ceph-mon[29330]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:57.170 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:56 smithi094 ceph-mon[33032]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:57.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:56 smithi111 ceph-mon[32784]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:57.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:56 smithi083 ceph-mon[29330]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:59.019 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:58 smithi094 ceph-mon[33032]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:59.019 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:58 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:59.019 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:58 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:59.019 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:37:58 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:59.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:58 smithi111 ceph-mon[32784]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:59.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:58 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:59.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:58 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:59.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:37:58 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:37:59.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:58 smithi083 ceph-mon[29330]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:37:59.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:58 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:59.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:58 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:37:59.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:37:58 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:01.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:00 smithi094 ceph-mon[33032]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:38:01.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:00 smithi083 ceph-mon[29330]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:38:01.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:01.186+0000 7faed8ff9700 1 -- 172.21.15.94:0/2265257544 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faedc062e80 con 0x7faec40a3c40 2024-04-25T11:38:01.175 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 4 on host 'smithi094' 2024-04-25T11:38:01.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:01.188+0000 7faee209b700 1 -- 172.21.15.94:0/2265257544 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faec40a3c40 msgr2=0x7faec40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:01.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:01.188+0000 7faee209b700 1 --2- 172.21.15.94:0/2265257544 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faec40a3c40 0x7faec40a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faecc005950 tx=0x7faecc00a300 comp rx=0 tx=0).stop 2024-04-25T11:38:01.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:01.188+0000 7faee209b700 1 -- 172.21.15.94:0/2265257544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc108820 msgr2=0x7faedc1a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:01.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:01.188+0000 7faee209b700 1 --2- 172.21.15.94:0/2265257544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc108820 0x7faedc1a0280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faed400b9e0 tx=0x7faed400bcf0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:01.188+0000 7faee209b700 1 -- 172.21.15.94:0/2265257544 shutdown_connections 2024-04-25T11:38:01.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:01.188+0000 7faee209b700 1 --2- 172.21.15.94:0/2265257544 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faec40a3c40 0x7faec40a6100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:01.188+0000 7faee209b700 1 --2- 172.21.15.94:0/2265257544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faedc108820 0x7faedc1a0280 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:01.188+0000 7faee209b700 1 --2- 172.21.15.94:0/2265257544 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faedc1035c0 0x7faedc198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:01.188+0000 7faee209b700 1 --2- 172.21.15.94:0/2265257544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faedc105ef0 0x7faedc199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:01.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:01.188+0000 7faee209b700 1 -- 172.21.15.94:0/2265257544 >> 172.21.15.94:0/2265257544 conn(0x7faedc0fcf70 msgr2=0x7faedc103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:01.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:01.189+0000 7faee209b700 1 -- 172.21.15.94:0/2265257544 shutdown_connections 2024-04-25T11:38:01.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:01.189+0000 7faee209b700 1 -- 172.21.15.94:0/2265257544 wait complete. 2024-04-25T11:38:01.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:00 smithi111 ceph-mon[32784]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:38:02.072 DEBUG:teuthology.orchestra.run.smithi094:osd.4> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T11:38:02.074 INFO:tasks.cephadm:Deploying osd.5 on smithi111 with /dev/vg_nvme/lv_3... 2024-04-25T11:38:02.074 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T11:38:02.319 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:38:02.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:02 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:02.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:02 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:02.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:02 smithi083 ceph-mon[29330]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:38:02.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:02 smithi083 ceph-mon[29330]: from='osd.4 [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T11:38:02.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:02 smithi083 ceph-mon[29330]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T11:38:02.440 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:02 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:02.440 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:02 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:02.440 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:02 smithi111 ceph-mon[32784]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:38:02.440 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:02 smithi111 ceph-mon[32784]: from='osd.4 [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T11:38:02.440 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:02 smithi111 ceph-mon[32784]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T11:38:02.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:02 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:02.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:02 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:02.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:02 smithi094 ceph-mon[33032]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:38:02.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:02 smithi094 ceph-mon[33032]: from='osd.4 [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T11:38:02.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:02 smithi094 ceph-mon[33032]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T11:38:03.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:03 smithi083 ceph-mon[29330]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T11:38:03.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:03 smithi083 ceph-mon[29330]: from='osd.4 [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:38:03.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:03 smithi083 ceph-mon[29330]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T11:38:03.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:03 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:38:03.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:03 smithi083 ceph-mon[29330]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:38:03.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:03 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:03.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:03 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:03.448 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:03 smithi094 ceph-mon[33032]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T11:38:03.448 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:03 smithi094 ceph-mon[33032]: from='osd.4 [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:38:03.448 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:03 smithi094 ceph-mon[33032]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T11:38:03.448 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:03 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:38:03.448 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:03 smithi094 ceph-mon[33032]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:38:03.448 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:03 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:03.448 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:03 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:03.449 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:38:03 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-4[44148]: 2024-04-25T11:38:03.160+0000 7f39a2e11700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T11:38:03.449 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:38:03 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-4[44148]: 2024-04-25T11:38:03.167+0000 7f399ec04700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:38:03.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:03 smithi111 ceph-mon[32784]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T11:38:03.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:03 smithi111 ceph-mon[32784]: from='osd.4 [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:38:03.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:03 smithi111 ceph-mon[32784]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T11:38:03.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:03 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:38:03.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:03 smithi111 ceph-mon[32784]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:38:03.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:03 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:03.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:03 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:04.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:04 smithi083 ceph-mon[29330]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T11:38:04.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:04 smithi083 ceph-mon[29330]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T11:38:04.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:38:04.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:04 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:38:04.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:04 smithi083 ceph-mon[29330]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:38:04.407 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:04 smithi111 ceph-mon[32784]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T11:38:04.407 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:04 smithi111 ceph-mon[32784]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T11:38:04.407 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:38:04.407 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:04 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:38:04.407 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:04 smithi111 ceph-mon[32784]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:38:04.407 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T11:38:04.407 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T11:38:04.407 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-25T11:38:04.408 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-25T11:38:04.408 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114109 s, 919 MB/s 2024-04-25T11:38:04.408 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-25T11:38:04.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:04 smithi094 ceph-mon[33032]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T11:38:04.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:04 smithi094 ceph-mon[33032]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T11:38:04.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:38:04.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:04 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:38:04.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:04 smithi094 ceph-mon[33032]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-04-25T11:38:05.068 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_3 2024-04-25T11:38:05.320 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:38:05.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:05 smithi111 ceph-mon[32784]: purged_snaps scrub starts 2024-04-25T11:38:05.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:05 smithi111 ceph-mon[32784]: purged_snaps scrub ok 2024-04-25T11:38:05.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:05 smithi111 ceph-mon[32784]: osd.4 [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394] boot 2024-04-25T11:38:05.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:05 smithi111 ceph-mon[32784]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T11:38:05.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:05 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:38:05.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:05 smithi083 ceph-mon[29330]: purged_snaps scrub starts 2024-04-25T11:38:05.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:05 smithi083 ceph-mon[29330]: purged_snaps scrub ok 2024-04-25T11:38:05.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:05 smithi083 ceph-mon[29330]: osd.4 [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394] boot 2024-04-25T11:38:05.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:05 smithi083 ceph-mon[29330]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T11:38:05.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:05 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:38:05.480 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:05 smithi094 ceph-mon[33032]: purged_snaps scrub starts 2024-04-25T11:38:05.480 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:05 smithi094 ceph-mon[33032]: purged_snaps scrub ok 2024-04-25T11:38:05.480 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:05 smithi094 ceph-mon[33032]: osd.4 [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394] boot 2024-04-25T11:38:05.480 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:05 smithi094 ceph-mon[33032]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T11:38:05.481 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:05 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:38:06.436 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T11:38:06.436 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: Detected new or changed devices on smithi094 2024-04-25T11:38:06.436 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:06.436 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:06.436 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:06.436 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:06.436 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi094 to 8544M 2024-04-25T11:38:06.436 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:06.436 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:06.436 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:06.437 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:06 smithi111 ceph-mon[32784]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:06.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T11:38:06.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: Detected new or changed devices on smithi094 2024-04-25T11:38:06.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:06.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:06.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:06.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:06.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi094 to 8544M 2024-04-25T11:38:06.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:06.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:06.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:06.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:06.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:06.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:06.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:06.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:06.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:06 smithi094 ceph-mon[33032]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:06.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T11:38:06.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: Detected new or changed devices on smithi094 2024-04-25T11:38:06.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:06.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:06.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:06.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:06.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi094 to 8544M 2024-04-25T11:38:06.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:06.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:06.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:06.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:06.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:06.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:06.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:06.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:06.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:06 smithi083 ceph-mon[29330]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:06.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.793+0000 7f228945e700 1 -- 172.21.15.111:0/2306233382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2284104680 msgr2=0x7f228410af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:06.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.793+0000 7f228945e700 1 --2- 172.21.15.111:0/2306233382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2284104680 0x7f228410af30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f227c00b0a0 tx=0x7f227c00b3b0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.793+0000 7f228945e700 1 -- 172.21.15.111:0/2306233382 shutdown_connections 2024-04-25T11:38:06.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.793+0000 7f228945e700 1 --2- 172.21.15.111:0/2306233382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2284104680 0x7f228410af30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.793+0000 7f228945e700 1 --2- 172.21.15.111:0/2306233382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2284102ae0 0x7f2284102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.793+0000 7f228945e700 1 --2- 172.21.15.111:0/2306233382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2284103ce0 0x7f2284104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.793+0000 7f228945e700 1 -- 172.21.15.111:0/2306233382 >> 172.21.15.111:0/2306233382 conn(0x7f22840fe080 msgr2=0x7f22841004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.793+0000 7f228945e700 1 -- 172.21.15.111:0/2306233382 shutdown_connections 2024-04-25T11:38:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.793+0000 7f228945e700 1 -- 172.21.15.111:0/2306233382 wait complete. 2024-04-25T11:38:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f228945e700 1 Processor -- start 2024-04-25T11:38:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f228945e700 1 -- start start 2024-04-25T11:38:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f228945e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2284102ae0 0x7f22840767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f228945e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2284103ce0 0x7f2284074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f228945e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2284104680 0x7f2284075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f228945e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f228410e580 con 0x7f2284104680 2024-04-25T11:38:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f228945e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f228410e400 con 0x7f2284103ce0 2024-04-25T11:38:06.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f228945e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f228410e700 con 0x7f2284102ae0 2024-04-25T11:38:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f2282ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2284102ae0 0x7f22840767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f2282ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2284102ae0 0x7f22840767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33256/0 (socket says 172.21.15.111:33256) 2024-04-25T11:38:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f2282ffd700 1 -- 172.21.15.111:0/2329110597 learned_addr learned my addr 172.21.15.111:0/2329110597 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-25T11:38:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f227bfff700 1 --2- 172.21.15.111:0/2329110597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2284103ce0 0x7f2284074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f22837fe700 1 --2- 172.21.15.111:0/2329110597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2284104680 0x7f2284075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f2282ffd700 1 -- 172.21.15.111:0/2329110597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2284103ce0 msgr2=0x7f2284074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f2282ffd700 1 --2- 172.21.15.111:0/2329110597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2284103ce0 0x7f2284074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f2282ffd700 1 -- 172.21.15.111:0/2329110597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2284104680 msgr2=0x7f2284075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.794+0000 7f2282ffd700 1 --2- 172.21.15.111:0/2329110597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2284104680 0x7f2284075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.795+0000 7f2282ffd700 1 -- 172.21.15.111:0/2329110597 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2284075c20 con 0x7f2284102ae0 2024-04-25T11:38:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.795+0000 7f2282ffd700 1 --2- 172.21.15.111:0/2329110597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2284102ae0 0x7f22840767e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f227400d980 tx=0x7f227400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.795+0000 7f2280ff9700 1 -- 172.21.15.111:0/2329110597 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274017030 con 0x7f2284102ae0 2024-04-25T11:38:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.795+0000 7f228945e700 1 -- 172.21.15.111:0/2329110597 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22841a2cd0 con 0x7f2284102ae0 2024-04-25T11:38:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.795+0000 7f2280ff9700 1 -- 172.21.15.111:0/2329110597 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22740044d0 con 0x7f2284102ae0 2024-04-25T11:38:06.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.795+0000 7f2280ff9700 1 -- 172.21.15.111:0/2329110597 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274003970 con 0x7f2284102ae0 2024-04-25T11:38:06.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.795+0000 7f228945e700 1 -- 172.21.15.111:0/2329110597 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22841a31c0 con 0x7f2284102ae0 2024-04-25T11:38:06.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.796+0000 7f228945e700 1 -- 172.21.15.111:0/2329110597 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2284069730 con 0x7f2284102ae0 2024-04-25T11:38:06.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.797+0000 7f2280ff9700 1 -- 172.21.15.111:0/2329110597 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f227401a070 con 0x7f2284102ae0 2024-04-25T11:38:06.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.797+0000 7f2280ff9700 1 --2- 172.21.15.111:0/2329110597 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f22700a3c40 0x7f22700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:06.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.797+0000 7f2280ff9700 1 -- 172.21.15.111:0/2329110597 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f22740cec40 con 0x7f2284102ae0 2024-04-25T11:38:06.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.798+0000 7f227bfff700 1 --2- 172.21.15.111:0/2329110597 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f22700a3c40 0x7f22700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:06.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.798+0000 7f227bfff700 1 --2- 172.21.15.111:0/2329110597 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f22700a3c40 0x7f22700a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f226c004630 tx=0x7f226c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:06.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:06.802+0000 7f2280ff9700 1 -- 172.21.15.111:0/2329110597 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f227409c690 con 0x7f2284102ae0 2024-04-25T11:38:06.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:07.008+0000 7f228945e700 1 -- 172.21.15.111:0/2329110597 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2284062e80 con 0x7f22700a3c40 2024-04-25T11:38:07.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:07 smithi111 ceph-mon[32784]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:38:07.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:07 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:38:07.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:07 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:38:07.938 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:07 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:08.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:07 smithi094 ceph-mon[33032]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:38:08.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:07 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:38:08.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:07 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:38:08.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:07 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:08.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:07 smithi083 ceph-mon[29330]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:38:08.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:07 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:38:08.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:07 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:38:08.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:07 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:08.783 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:08 smithi111 ceph-mon[32784]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:09.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:08 smithi094 ceph-mon[33032]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:09.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:08 smithi083 ceph-mon[29330]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:10.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:09 smithi094 ceph-mon[33032]: from='client.? 172.21.15.111:0/1074471604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "050ab73c-e5d2-4140-8118-8f7c65b40229"}]: dispatch 2024-04-25T11:38:10.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:09 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "050ab73c-e5d2-4140-8118-8f7c65b40229"}]: dispatch 2024-04-25T11:38:10.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:09 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "050ab73c-e5d2-4140-8118-8f7c65b40229"}]': finished 2024-04-25T11:38:10.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:09 smithi094 ceph-mon[33032]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T11:38:10.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:09 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:38:10.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:09 smithi111 ceph-mon[32784]: from='client.? 172.21.15.111:0/1074471604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "050ab73c-e5d2-4140-8118-8f7c65b40229"}]: dispatch 2024-04-25T11:38:10.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:09 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "050ab73c-e5d2-4140-8118-8f7c65b40229"}]: dispatch 2024-04-25T11:38:10.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:09 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "050ab73c-e5d2-4140-8118-8f7c65b40229"}]': finished 2024-04-25T11:38:10.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:09 smithi111 ceph-mon[32784]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T11:38:10.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:09 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:38:10.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:09 smithi083 ceph-mon[29330]: from='client.? 172.21.15.111:0/1074471604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "050ab73c-e5d2-4140-8118-8f7c65b40229"}]: dispatch 2024-04-25T11:38:10.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:09 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "050ab73c-e5d2-4140-8118-8f7c65b40229"}]: dispatch 2024-04-25T11:38:10.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:09 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "050ab73c-e5d2-4140-8118-8f7c65b40229"}]': finished 2024-04-25T11:38:10.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:09 smithi083 ceph-mon[29330]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T11:38:10.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:09 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:38:11.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:10 smithi111 ceph-mon[32784]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:11.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:10 smithi111 ceph-mon[32784]: from='client.? 172.21.15.111:0/2395693300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:38:11.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:10 smithi083 ceph-mon[29330]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:11.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:10 smithi083 ceph-mon[29330]: from='client.? 172.21.15.111:0/2395693300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:38:11.173 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:10 smithi094 ceph-mon[33032]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:11.173 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:10 smithi094 ceph-mon[33032]: from='client.? 172.21.15.111:0/2395693300' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:38:13.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:12 smithi083 ceph-mon[29330]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:13.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:12 smithi094 ceph-mon[33032]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:13.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:12 smithi111 ceph-mon[32784]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:15.176 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:14 smithi111 ceph-mon[32784]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:15.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:14 smithi094 ceph-mon[33032]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:15.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:14 smithi083 ceph-mon[29330]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:17.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:16 smithi094 ceph-mon[33032]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:17.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:16 smithi111 ceph-mon[32784]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:17.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:16 smithi083 ceph-mon[29330]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:17.963 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:17 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T11:38:17.963 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:17 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:18.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:17 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T11:38:18.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:17 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:18.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:17 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T11:38:18.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:17 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:19.185 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:18 smithi111 ceph-mon[32784]: Deploying daemon osd.5 on smithi111 2024-04-25T11:38:19.185 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:18 smithi111 ceph-mon[32784]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:19.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:18 smithi094 ceph-mon[33032]: Deploying daemon osd.5 on smithi111 2024-04-25T11:38:19.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:18 smithi094 ceph-mon[33032]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:19.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:18 smithi083 ceph-mon[29330]: Deploying daemon osd.5 on smithi111 2024-04-25T11:38:19.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:18 smithi083 ceph-mon[29330]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:21.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:20 smithi111 ceph-mon[32784]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:21.208 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:20 smithi083 ceph-mon[29330]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:21.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:20 smithi094 ceph-mon[33032]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:23.090 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:22 smithi111 ceph-mon[32784]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:23.090 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:22 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:23.090 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:22 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:23.090 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:22 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:23.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:22 smithi094 ceph-mon[33032]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:23.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:22 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:23.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:22 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:23.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:22 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:23.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:22 smithi083 ceph-mon[29330]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:23.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:22 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:23.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:22 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:23.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:22 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:25.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:24 smithi094 ceph-mon[33032]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:25.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:24 smithi111 ceph-mon[32784]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:25.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:24 smithi083 ceph-mon[29330]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:25.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:25.873+0000 7f2280ff9700 1 -- 172.21.15.111:0/2329110597 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2284062e80 con 0x7f22700a3c40 2024-04-25T11:38:25.861 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 5 on host 'smithi111' 2024-04-25T11:38:25.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:25.877+0000 7f228945e700 1 -- 172.21.15.111:0/2329110597 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f22700a3c40 msgr2=0x7f22700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:25.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:25.877+0000 7f228945e700 1 --2- 172.21.15.111:0/2329110597 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f22700a3c40 0x7f22700a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f226c004630 tx=0x7f226c00b3c0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:25.877+0000 7f228945e700 1 -- 172.21.15.111:0/2329110597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2284102ae0 msgr2=0x7f22840767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:25.877+0000 7f228945e700 1 --2- 172.21.15.111:0/2329110597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2284102ae0 0x7f22840767e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f227400d980 tx=0x7f227400dc90 comp rx=0 tx=0).stop 2024-04-25T11:38:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:25.877+0000 7f228945e700 1 -- 172.21.15.111:0/2329110597 shutdown_connections 2024-04-25T11:38:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:25.877+0000 7f228945e700 1 --2- 172.21.15.111:0/2329110597 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f22700a3c40 0x7f22700a6100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:25.877+0000 7f228945e700 1 --2- 172.21.15.111:0/2329110597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2284104680 0x7f2284075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:25.877+0000 7f228945e700 1 --2- 172.21.15.111:0/2329110597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2284102ae0 0x7f22840767e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:25.877+0000 7f228945e700 1 --2- 172.21.15.111:0/2329110597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2284103ce0 0x7f2284074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:25.877+0000 7f228945e700 1 -- 172.21.15.111:0/2329110597 >> 172.21.15.111:0/2329110597 conn(0x7f22840fe080 msgr2=0x7f2284106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:25.877+0000 7f228945e700 1 -- 172.21.15.111:0/2329110597 shutdown_connections 2024-04-25T11:38:25.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:38:25.877+0000 7f228945e700 1 -- 172.21.15.111:0/2329110597 wait complete. 2024-04-25T11:38:26.528 DEBUG:teuthology.orchestra.run.smithi111:osd.5> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T11:38:26.530 INFO:tasks.cephadm:Deploying osd.6 on smithi083 with /dev/vg_nvme/lv_2... 2024-04-25T11:38:26.531 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T11:38:26.775 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:38:27.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:26 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:27.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:26 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:27.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:26 smithi111 ceph-mon[32784]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:27.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:26 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:27.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:26 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:27.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:26 smithi111 ceph-mon[32784]: from='osd.5 [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T11:38:27.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:26 smithi111 ceph-mon[32784]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T11:38:27.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:26 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:27.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:26 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:27.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:26 smithi083 ceph-mon[29330]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:27.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:26 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:27.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:26 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:27.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:26 smithi083 ceph-mon[29330]: from='osd.5 [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T11:38:27.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:26 smithi083 ceph-mon[29330]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T11:38:27.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:26 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:27.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:26 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:27.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:26 smithi094 ceph-mon[33032]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:27.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:26 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:27.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:26 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:27.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:26 smithi094 ceph-mon[33032]: from='osd.5 [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T11:38:27.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:26 smithi094 ceph-mon[33032]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T11:38:28.144 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:27 smithi083 ceph-mon[29330]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T11:38:28.144 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:27 smithi083 ceph-mon[29330]: from='osd.5 [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:38:28.144 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:27 smithi083 ceph-mon[29330]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T11:38:28.144 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:27 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:38:28.144 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:27 smithi083 ceph-mon[29330]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:38:28.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:27 smithi094 ceph-mon[33032]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T11:38:28.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:27 smithi094 ceph-mon[33032]: from='osd.5 [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:38:28.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:27 smithi094 ceph-mon[33032]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T11:38:28.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:27 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:38:28.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:27 smithi094 ceph-mon[33032]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:38:28.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:27 smithi111 ceph-mon[32784]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T11:38:28.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:27 smithi111 ceph-mon[32784]: from='osd.5 [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:38:28.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:27 smithi111 ceph-mon[32784]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T11:38:28.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:27 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:38:28.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:27 smithi111 ceph-mon[32784]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:38:28.323 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:38:27 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:38:27.986+0000 7f8c589dc700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T11:38:28.323 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:38:27 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:38:27.993+0000 7f8c547cf700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:38:28.696 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T11:38:28.697 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T11:38:28.697 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-25T11:38:28.697 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-25T11:38:28.697 INFO:teuthology.orchestra.run.smithi083.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113028 s, 928 MB/s 2024-04-25T11:38:28.697 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 2024-04-25T11:38:28.697 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-25T11:38:29.233 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:28 smithi111 ceph-mon[32784]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:29.233 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:28 smithi111 ceph-mon[32784]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-25T11:38:29.233 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:28 smithi111 ceph-mon[32784]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T11:38:29.233 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:28 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:38:29.234 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:28 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:38:29.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:28 smithi083 ceph-mon[29330]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:29.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:28 smithi083 ceph-mon[29330]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-25T11:38:29.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:28 smithi083 ceph-mon[29330]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T11:38:29.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:28 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:38:29.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:28 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:38:29.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:28 smithi094 ceph-mon[33032]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 139 MiB used, 447 GiB / 447 GiB avail 2024-04-25T11:38:29.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:28 smithi094 ceph-mon[33032]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-25T11:38:29.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:28 smithi094 ceph-mon[33032]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T11:38:29.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:28 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:38:29.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:28 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:38:29.287 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_2 2024-04-25T11:38:29.554 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:38:30.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: purged_snaps scrub starts 2024-04-25T11:38:30.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: purged_snaps scrub ok 2024-04-25T11:38:30.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: osd.5 [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490] boot 2024-04-25T11:38:30.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T11:38:30.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:38:30.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:30.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:30.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:30.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:30.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:30.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:30.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:30.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:30.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:30.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:30.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:30.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:29 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:30.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: purged_snaps scrub starts 2024-04-25T11:38:30.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: purged_snaps scrub ok 2024-04-25T11:38:30.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: osd.5 [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490] boot 2024-04-25T11:38:30.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T11:38:30.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:38:30.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:30.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:30.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:30.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:30.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:30.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:30.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:30.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:30.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:30.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:30.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:30.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:29 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: purged_snaps scrub starts 2024-04-25T11:38:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: purged_snaps scrub ok 2024-04-25T11:38:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: osd.5 [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490] boot 2024-04-25T11:38:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T11:38:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:38:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:30.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:30.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:30.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:30.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:30.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:30.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:30.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:30.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:29 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:31.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.077+0000 7fdf1a9b8700 1 -- 172.21.15.83:0/984679055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf14101170 msgr2=0x7fdf14101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.077+0000 7fdf1a9b8700 1 --2- 172.21.15.83:0/984679055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf14101170 0x7fdf14101590 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdf040099c0 tx=0x7fdf04009cd0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.077+0000 7fdf1a9b8700 1 -- 172.21.15.83:0/984679055 shutdown_connections 2024-04-25T11:38:31.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.077+0000 7fdf1a9b8700 1 --2- 172.21.15.83:0/984679055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf14102d10 0x7fdf141095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.077+0000 7fdf1a9b8700 1 --2- 172.21.15.83:0/984679055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf14101170 0x7fdf14101590 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.077+0000 7fdf1a9b8700 1 --2- 172.21.15.83:0/984679055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf14102370 0x7fdf141027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.077+0000 7fdf1a9b8700 1 -- 172.21.15.83:0/984679055 >> 172.21.15.83:0/984679055 conn(0x7fdf140fc710 msgr2=0x7fdf140feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:31.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.077+0000 7fdf1a9b8700 1 -- 172.21.15.83:0/984679055 shutdown_connections 2024-04-25T11:38:31.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.077+0000 7fdf1a9b8700 1 -- 172.21.15.83:0/984679055 wait complete. 2024-04-25T11:38:31.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.078+0000 7fdf1a9b8700 1 Processor -- start 2024-04-25T11:38:31.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.078+0000 7fdf1a9b8700 1 -- start start 2024-04-25T11:38:31.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.078+0000 7fdf1a9b8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf14101170 0x7fdf1410ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.078+0000 7fdf1a9b8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf14102370 0x7fdf1419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.078+0000 7fdf1a9b8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf14102d10 0x7fdf1419ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.078+0000 7fdf1a9b8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1410d4c0 con 0x7fdf14101170 2024-04-25T11:38:31.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.078+0000 7fdf1a9b8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1410d340 con 0x7fdf14102370 2024-04-25T11:38:31.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.078+0000 7fdf1a9b8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1410d640 con 0x7fdf14102d10 2024-04-25T11:38:31.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.078+0000 7fdf13fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf14101170 0x7fdf1410ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.078+0000 7fdf13fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf14101170 0x7fdf1410ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44390/0 (socket says 172.21.15.83:44390) 2024-04-25T11:38:31.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.079+0000 7fdf13fff700 1 -- 172.21.15.83:0/2744260800 learned_addr learned my addr 172.21.15.83:0/2744260800 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:38:31.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.079+0000 7fdf137fe700 1 --2- 172.21.15.83:0/2744260800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf14102370 0x7fdf1419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.079+0000 7fdf18f55700 1 --2- 172.21.15.83:0/2744260800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf14102d10 0x7fdf1419ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.079+0000 7fdf13fff700 1 -- 172.21.15.83:0/2744260800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf14102d10 msgr2=0x7fdf1419ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.079+0000 7fdf13fff700 1 --2- 172.21.15.83:0/2744260800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf14102d10 0x7fdf1419ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.079+0000 7fdf13fff700 1 -- 172.21.15.83:0/2744260800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf14102370 msgr2=0x7fdf1419d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:31.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.079+0000 7fdf13fff700 1 --2- 172.21.15.83:0/2744260800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf14102370 0x7fdf1419d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:31.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.079+0000 7fdf13fff700 1 -- 172.21.15.83:0/2744260800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf141a0890 con 0x7fdf14101170 2024-04-25T11:38:31.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.079+0000 7fdf137fe700 1 --2- 172.21.15.83:0/2744260800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf14102370 0x7fdf1419d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:38:31.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.079+0000 7fdf13fff700 1 --2- 172.21.15.83:0/2744260800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf14101170 0x7fdf1410ab30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400fa00 tx=0x7fdf0400fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:31.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.079+0000 7fdf117fa700 1 -- 172.21.15.83:0/2744260800 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04027070 con 0x7fdf14101170 2024-04-25T11:38:31.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.079+0000 7fdf117fa700 1 -- 172.21.15.83:0/2744260800 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf04005940 con 0x7fdf14101170 2024-04-25T11:38:31.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.079+0000 7fdf1a9b8700 1 -- 172.21.15.83:0/2744260800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf141a0b20 con 0x7fdf14101170 2024-04-25T11:38:31.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.079+0000 7fdf117fa700 1 -- 172.21.15.83:0/2744260800 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf040173f0 con 0x7fdf14101170 2024-04-25T11:38:31.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.079+0000 7fdf1a9b8700 1 -- 172.21.15.83:0/2744260800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf141a1000 con 0x7fdf14101170 2024-04-25T11:38:31.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.080+0000 7fdf1a9b8700 1 -- 172.21.15.83:0/2744260800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf14069730 con 0x7fdf14101170 2024-04-25T11:38:31.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.081+0000 7fdf117fa700 1 -- 172.21.15.83:0/2744260800 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdf0402a070 con 0x7fdf14101170 2024-04-25T11:38:31.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.082+0000 7fdf117fa700 1 --2- 172.21.15.83:0/2744260800 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdefc0a3c40 0x7fdefc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:31.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.082+0000 7fdf137fe700 1 --2- 172.21.15.83:0/2744260800 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdefc0a3c40 0x7fdefc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:31.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.082+0000 7fdf117fa700 1 -- 172.21.15.83:0/2744260800 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7fdf040d0090 con 0x7fdf14101170 2024-04-25T11:38:31.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.082+0000 7fdf137fe700 1 --2- 172.21.15.83:0/2744260800 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdefc0a3c40 0x7fdefc0a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf1410b0b0 tx=0x7fdf08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:31.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.087+0000 7fdf117fa700 1 -- 172.21.15.83:0/2744260800 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf04013070 con 0x7fdf14101170 2024-04-25T11:38:31.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:30 smithi083 ceph-mon[29330]: Detected new or changed devices on smithi111 2024-04-25T11:38:31.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:30 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi111 to 8544M 2024-04-25T11:38:31.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:30 smithi083 ceph-mon[29330]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:31.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:30 smithi083 ceph-mon[29330]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T11:38:31.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:30 smithi094 ceph-mon[33032]: Detected new or changed devices on smithi111 2024-04-25T11:38:31.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:30 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi111 to 8544M 2024-04-25T11:38:31.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:30 smithi094 ceph-mon[33032]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:31.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:30 smithi094 ceph-mon[33032]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T11:38:31.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:31.295+0000 7fdf1a9b8700 1 -- 172.21.15.83:0/2744260800 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdf14062e80 con 0x7fdefc0a3c40 2024-04-25T11:38:31.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:31 smithi111 ceph-mon[32784]: Detected new or changed devices on smithi111 2024-04-25T11:38:31.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:31 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi111 to 8544M 2024-04-25T11:38:31.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:31 smithi111 ceph-mon[32784]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:31.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:31 smithi111 ceph-mon[32784]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T11:38:32.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:31 smithi083 ceph-mon[29330]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T11:38:32.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:32 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:38:32.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:32 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:38:32.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:32 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:32.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:32 smithi083 ceph-mon[29330]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T11:38:32.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:32 smithi094 ceph-mon[33032]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T11:38:32.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:32 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:38:32.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:32 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:38:32.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:32 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:32.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:32 smithi094 ceph-mon[33032]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T11:38:32.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:32 smithi111 ceph-mon[32784]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T11:38:32.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:32 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:38:32.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:32 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:38:32.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:32 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:32.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:32 smithi111 ceph-mon[32784]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T11:38:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:33 smithi083 ceph-mon[29330]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:38:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:33 smithi083 ceph-mon[29330]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:33.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:33 smithi083 ceph-mon[29330]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T11:38:33.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:33 smithi094 ceph-mon[33032]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:38:33.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:33 smithi094 ceph-mon[33032]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:33.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:33 smithi094 ceph-mon[33032]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T11:38:33.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:33 smithi111 ceph-mon[32784]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:38:33.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:33 smithi111 ceph-mon[32784]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:33.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:33 smithi111 ceph-mon[32784]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T11:38:34.254 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:34 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1265713278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44e6b825-a33f-447c-bc4c-f1890016bd3e"}]: dispatch 2024-04-25T11:38:34.255 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:34 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1265713278' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44e6b825-a33f-447c-bc4c-f1890016bd3e"}]': finished 2024-04-25T11:38:34.255 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:34 smithi083 ceph-mon[29330]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T11:38:34.255 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:34.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:34 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1265713278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44e6b825-a33f-447c-bc4c-f1890016bd3e"}]: dispatch 2024-04-25T11:38:34.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:34 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1265713278' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44e6b825-a33f-447c-bc4c-f1890016bd3e"}]': finished 2024-04-25T11:38:34.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:34 smithi094 ceph-mon[33032]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T11:38:34.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:34.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:34 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1265713278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44e6b825-a33f-447c-bc4c-f1890016bd3e"}]: dispatch 2024-04-25T11:38:34.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:34 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1265713278' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44e6b825-a33f-447c-bc4c-f1890016bd3e"}]': finished 2024-04-25T11:38:34.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:34 smithi111 ceph-mon[32784]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T11:38:34.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:35.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:35 smithi094 ceph-mon[33032]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T11:38:35.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:35 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/4160021735' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:38:35.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:35 smithi111 ceph-mon[32784]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T11:38:35.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:35 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/4160021735' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:38:35.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:35 smithi083 ceph-mon[29330]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T11:38:35.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:35 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/4160021735' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:38:37.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:37 smithi094 ceph-mon[33032]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-25T11:38:37.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:37 smithi111 ceph-mon[32784]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-25T11:38:37.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:37 smithi083 ceph-mon[29330]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 167 MiB used, 536 GiB / 536 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-25T11:38:39.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:39 smithi094 ceph-mon[33032]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-25T11:38:39.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:39 smithi111 ceph-mon[32784]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-25T11:38:39.334 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:39 smithi083 ceph-mon[29330]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-25T11:38:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:41 smithi094 ceph-mon[33032]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T11:38:41.316 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:41 smithi083 ceph-mon[29330]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T11:38:41.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:41 smithi111 ceph-mon[32784]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T11:38:42.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:42 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T11:38:42.253 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:42 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:42.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:42 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T11:38:42.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:42 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:42.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:42 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T11:38:42.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:42 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:43.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:43 smithi094 ceph-mon[33032]: Deploying daemon osd.6 on smithi083 2024-04-25T11:38:43.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:43 smithi094 ceph-mon[33032]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-25T11:38:43.287 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:43 smithi083 ceph-mon[29330]: Deploying daemon osd.6 on smithi083 2024-04-25T11:38:43.287 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:43 smithi083 ceph-mon[29330]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-25T11:38:43.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:43 smithi111 ceph-mon[32784]: Deploying daemon osd.6 on smithi083 2024-04-25T11:38:43.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:43 smithi111 ceph-mon[32784]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-25T11:38:45.246 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:45 smithi083 ceph-mon[29330]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:45 smithi094 ceph-mon[33032]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:45 smithi111 ceph-mon[32784]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:47.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:47 smithi094 ceph-mon[33032]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:47.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:47 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:47.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:47 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:47.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:47 smithi111 ceph-mon[32784]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:47.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:47 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:47.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:47 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:47.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:47 smithi083 ceph-mon[29330]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:47.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:47 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:47.402 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:47 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:48.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:48 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:48.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:48 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:48.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:48 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:49.119 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:49 smithi083 ceph-mon[29330]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:49.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:49 smithi094 ceph-mon[33032]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:49.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:49 smithi111 ceph-mon[32784]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:50.714+0000 7fdf117fa700 1 -- 172.21.15.83:0/2744260800 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdf14062e80 con 0x7fdefc0a3c40 2024-04-25T11:38:50.701 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 6 on host 'smithi083' 2024-04-25T11:38:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:50.716+0000 7fdf1a9b8700 1 -- 172.21.15.83:0/2744260800 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdefc0a3c40 msgr2=0x7fdefc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:50.716+0000 7fdf1a9b8700 1 --2- 172.21.15.83:0/2744260800 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdefc0a3c40 0x7fdefc0a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf1410b0b0 tx=0x7fdf08009260 comp rx=0 tx=0).stop 2024-04-25T11:38:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:50.716+0000 7fdf1a9b8700 1 -- 172.21.15.83:0/2744260800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf14101170 msgr2=0x7fdf1410ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:50.716+0000 7fdf1a9b8700 1 --2- 172.21.15.83:0/2744260800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf14101170 0x7fdf1410ab30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400fa00 tx=0x7fdf0400fa30 comp rx=0 tx=0).stop 2024-04-25T11:38:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:50.716+0000 7fdf1a9b8700 1 -- 172.21.15.83:0/2744260800 shutdown_connections 2024-04-25T11:38:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:50.716+0000 7fdf1a9b8700 1 --2- 172.21.15.83:0/2744260800 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdefc0a3c40 0x7fdefc0a6100 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:50.716+0000 7fdf1a9b8700 1 --2- 172.21.15.83:0/2744260800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf14102d10 0x7fdf1419ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:50.716+0000 7fdf1a9b8700 1 --2- 172.21.15.83:0/2744260800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf14101170 0x7fdf1410ab30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:50.716+0000 7fdf1a9b8700 1 --2- 172.21.15.83:0/2744260800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf14102370 0x7fdf1419d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:50.716+0000 7fdf1a9b8700 1 -- 172.21.15.83:0/2744260800 >> 172.21.15.83:0/2744260800 conn(0x7fdf140fc710 msgr2=0x7fdf141056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:50.716+0000 7fdf1a9b8700 1 -- 172.21.15.83:0/2744260800 shutdown_connections 2024-04-25T11:38:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:38:50.716+0000 7fdf1a9b8700 1 -- 172.21.15.83:0/2744260800 wait complete. 2024-04-25T11:38:50.981 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:50.981 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:50.982 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:50.982 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:50.982 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:50.982 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:50 smithi083 ceph-mon[29330]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:50.982 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:50.982 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:50.982 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:51.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:51.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:51.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:51.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:51.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:51.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:50 smithi111 ceph-mon[32784]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:51.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:51.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:51.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:51.265 DEBUG:teuthology.orchestra.run.smithi083:osd.6> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T11:38:51.268 INFO:tasks.cephadm:Deploying osd.7 on smithi094 with /dev/vg_nvme/lv_2... 2024-04-25T11:38:51.268 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T11:38:51.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:51.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:51.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:51.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:51.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:51.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:50 smithi094 ceph-mon[33032]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:51.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:51.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:51.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:38:51.515 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:38:51.965 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:51 smithi094 ceph-mon[33032]: from='osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T11:38:52.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:51 smithi111 ceph-mon[32784]: from='osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T11:38:52.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:51 smithi083 ceph-mon[29330]: from='osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T11:38:52.977 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:52 smithi094 ceph-mon[33032]: from='osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T11:38:52.977 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:52 smithi094 ceph-mon[33032]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T11:38:52.977 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:52 smithi094 ceph-mon[33032]: from='osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T11:38:52.977 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:52.978 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:52 smithi094 ceph-mon[33032]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:52 smithi083 ceph-mon[29330]: from='osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T11:38:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:52 smithi083 ceph-mon[29330]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T11:38:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:52 smithi083 ceph-mon[29330]: from='osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T11:38:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:53.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:52 smithi083 ceph-mon[29330]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:53.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:52 smithi111 ceph-mon[32784]: from='osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T11:38:53.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:52 smithi111 ceph-mon[32784]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T11:38:53.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:52 smithi111 ceph-mon[32784]: from='osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T11:38:53.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:53.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:52 smithi111 ceph-mon[32784]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:53.761 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T11:38:53.761 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T11:38:53.761 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-25T11:38:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-25T11:38:53.762 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00983564 s, 1.1 GB/s 2024-04-25T11:38:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-25T11:38:53.861 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:38:53 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:38:53.739+0000 7fbb3bc00700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T11:38:53.861 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:38:53 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:38:53.761+0000 7fbb331ea700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:38:54.116 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:53 smithi094 ceph-mon[33032]: from='osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T11:38:54.117 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:53 smithi094 ceph-mon[33032]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T11:38:54.117 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:53 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:54.117 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:53 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:54.117 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:53 smithi094 ceph-mon[33032]: from='osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773]' entity='osd.6' 2024-04-25T11:38:54.117 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:53 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:54.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:53 smithi083 ceph-mon[29330]: from='osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T11:38:54.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:53 smithi083 ceph-mon[29330]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T11:38:54.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:53 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:54.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:53 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:54.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:53 smithi083 ceph-mon[29330]: from='osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773]' entity='osd.6' 2024-04-25T11:38:54.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:53 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:54.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:53 smithi111 ceph-mon[32784]: from='osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T11:38:54.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:53 smithi111 ceph-mon[32784]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T11:38:54.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:53 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:54.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:53 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:54.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:53 smithi111 ceph-mon[32784]: from='osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773]' entity='osd.6' 2024-04-25T11:38:54.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:53 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:54.451 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_2 2024-04-25T11:38:54.727 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:38:55.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:54 smithi083 ceph-mon[29330]: purged_snaps scrub starts 2024-04-25T11:38:55.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:54 smithi083 ceph-mon[29330]: purged_snaps scrub ok 2024-04-25T11:38:55.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:54 smithi083 ceph-mon[29330]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:55.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:55.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:55.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:55.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:55.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:55.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:55.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:55.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:55.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:55.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:54 smithi083 ceph-mon[29330]: osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773] boot 2024-04-25T11:38:55.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:54 smithi083 ceph-mon[29330]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T11:38:55.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:55.169 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:54 smithi094 ceph-mon[33032]: purged_snaps scrub starts 2024-04-25T11:38:55.170 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:54 smithi094 ceph-mon[33032]: purged_snaps scrub ok 2024-04-25T11:38:55.170 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:54 smithi094 ceph-mon[33032]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:55.170 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:55.170 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:55.170 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:55.170 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:55.170 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:55.170 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:55.170 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:55.170 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:55.170 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:55.170 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:54 smithi094 ceph-mon[33032]: osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773] boot 2024-04-25T11:38:55.171 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:54 smithi094 ceph-mon[33032]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T11:38:55.171 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:55.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:54 smithi111 ceph-mon[32784]: purged_snaps scrub starts 2024-04-25T11:38:55.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:54 smithi111 ceph-mon[32784]: purged_snaps scrub ok 2024-04-25T11:38:55.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:54 smithi111 ceph-mon[32784]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T11:38:55.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:55.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:55.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:55.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:55.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:38:55.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:55.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:55.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:38:55.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:38:55.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:54 smithi111 ceph-mon[32784]: osd.6 [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773] boot 2024-04-25T11:38:55.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:54 smithi111 ceph-mon[32784]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T11:38:55.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:38:56.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:55 smithi083 ceph-mon[29330]: Detected new or changed devices on smithi083 2024-04-25T11:38:56.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:55 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-25T11:38:56.172 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:55 smithi094 ceph-mon[33032]: Detected new or changed devices on smithi083 2024-04-25T11:38:56.172 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:55 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-25T11:38:56.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:55 smithi111 ceph-mon[32784]: Detected new or changed devices on smithi083 2024-04-25T11:38:56.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:55 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-25T11:38:56.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.408+0000 7f850e092700 1 -- 172.21.15.94:0/1045622822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85081019b0 msgr2=0x7f8508101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.408+0000 7f850e092700 1 --2- 172.21.15.94:0/1045622822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85081019b0 0x7f8508101dd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f84f80099c0 tx=0x7f84f8009cd0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.408+0000 7f850e092700 1 -- 172.21.15.94:0/1045622822 shutdown_connections 2024-04-25T11:38:56.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.408+0000 7f850e092700 1 --2- 172.21.15.94:0/1045622822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8508103550 0x7f8508109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.408+0000 7f850e092700 1 --2- 172.21.15.94:0/1045622822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85081019b0 0x7f8508101dd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.408+0000 7f850e092700 1 --2- 172.21.15.94:0/1045622822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8508102bb0 0x7f8508103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.408+0000 7f850e092700 1 -- 172.21.15.94:0/1045622822 >> 172.21.15.94:0/1045622822 conn(0x7f85080fcf70 msgr2=0x7f85080ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:38:56.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.408+0000 7f850e092700 1 -- 172.21.15.94:0/1045622822 shutdown_connections 2024-04-25T11:38:56.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.408+0000 7f850e092700 1 -- 172.21.15.94:0/1045622822 wait complete. 2024-04-25T11:38:56.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.409+0000 7f850e092700 1 Processor -- start 2024-04-25T11:38:56.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.409+0000 7f850e092700 1 -- start start 2024-04-25T11:38:56.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.409+0000 7f850e092700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85081019b0 0x7f8508197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.409+0000 7f850e092700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8508102bb0 0x7f8508198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.409+0000 7f850e092700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8508103550 0x7f850819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.409+0000 7f850e092700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f850810d4a0 con 0x7f8508103550 2024-04-25T11:38:56.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.409+0000 7f850e092700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f850810d320 con 0x7f85081019b0 2024-04-25T11:38:56.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.409+0000 7f850e092700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f850810d620 con 0x7f8508102bb0 2024-04-25T11:38:56.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.409+0000 7f85077fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85081019b0 0x7f8508197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:56.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.409+0000 7f85077fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85081019b0 0x7f8508197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45384/0 (socket says 172.21.15.94:45384) 2024-04-25T11:38:56.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.409+0000 7f85077fe700 1 -- 172.21.15.94:0/2548036342 learned_addr learned my addr 172.21.15.94:0/2548036342 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T11:38:56.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.409+0000 7f8506ffd700 1 --2- 172.21.15.94:0/2548036342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8508102bb0 0x7f8508198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:56.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.409+0000 7f8507fff700 1 --2- 172.21.15.94:0/2548036342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8508103550 0x7f850819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:56.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.410+0000 7f85077fe700 1 -- 172.21.15.94:0/2548036342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8508102bb0 msgr2=0x7f8508198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.410+0000 7f85077fe700 1 --2- 172.21.15.94:0/2548036342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8508102bb0 0x7f8508198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.410+0000 7f85077fe700 1 -- 172.21.15.94:0/2548036342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8508103550 msgr2=0x7f850819f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:38:56.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.410+0000 7f85077fe700 1 --2- 172.21.15.94:0/2548036342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8508103550 0x7f850819f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:38:56.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.410+0000 7f85077fe700 1 -- 172.21.15.94:0/2548036342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850819f890 con 0x7f85081019b0 2024-04-25T11:38:56.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.410+0000 7f85077fe700 1 --2- 172.21.15.94:0/2548036342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85081019b0 0x7f8508197c40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f84f8006010 tx=0x7f84f8005af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:56.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.410+0000 7f8504ff9700 1 -- 172.21.15.94:0/2548036342 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8027070 con 0x7f85081019b0 2024-04-25T11:38:56.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.410+0000 7f850e092700 1 -- 172.21.15.94:0/2548036342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850819fb20 con 0x7f85081019b0 2024-04-25T11:38:56.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.410+0000 7f8504ff9700 1 -- 172.21.15.94:0/2548036342 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84f8004120 con 0x7f85081019b0 2024-04-25T11:38:56.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.410+0000 7f8504ff9700 1 -- 172.21.15.94:0/2548036342 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8022e40 con 0x7f85081019b0 2024-04-25T11:38:56.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.410+0000 7f850e092700 1 -- 172.21.15.94:0/2548036342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85081a0050 con 0x7f85081019b0 2024-04-25T11:38:56.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.411+0000 7f850e092700 1 -- 172.21.15.94:0/2548036342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8508069730 con 0x7f85081019b0 2024-04-25T11:38:56.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.412+0000 7f8504ff9700 1 -- 172.21.15.94:0/2548036342 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f84f802a070 con 0x7f85081019b0 2024-04-25T11:38:56.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.413+0000 7f8504ff9700 1 --2- 172.21.15.94:0/2548036342 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84f00a3c40 0x7f84f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:38:56.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.413+0000 7f8504ff9700 1 -- 172.21.15.94:0/2548036342 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f84f80d0600 con 0x7f85081019b0 2024-04-25T11:38:56.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.413+0000 7f8506ffd700 1 --2- 172.21.15.94:0/2548036342 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84f00a3c40 0x7f84f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:38:56.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.413+0000 7f8506ffd700 1 --2- 172.21.15.94:0/2548036342 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84f00a3c40 0x7f84f00a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f84fc006fd0 tx=0x7f84fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:38:56.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.417+0000 7f8504ff9700 1 -- 172.21.15.94:0/2548036342 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84f8013070 con 0x7f85081019b0 2024-04-25T11:38:56.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:38:56.623+0000 7f850e092700 1 -- 172.21.15.94:0/2548036342 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f850806b190 con 0x7f84f00a3c40 2024-04-25T11:38:57.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:56 smithi083 ceph-mon[29330]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:38:57.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:56 smithi083 ceph-mon[29330]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T11:38:57.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:56 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:38:57.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:56 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:38:57.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:56 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:57.218 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:56 smithi094 ceph-mon[33032]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:38:57.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:56 smithi094 ceph-mon[33032]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T11:38:57.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:56 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:38:57.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:56 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:38:57.219 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:56 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:57.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:56 smithi111 ceph-mon[32784]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:38:57.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:56 smithi111 ceph-mon[32784]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T11:38:57.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:56 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:38:57.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:56 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:38:57.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:56 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:38:58.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:57 smithi083 ceph-mon[29330]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:38:58.232 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:57 smithi094 ceph-mon[33032]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:38:58.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:57 smithi111 ceph-mon[32784]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:38:59.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:58 smithi083 ceph-mon[29330]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:38:59.223 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:58 smithi111 ceph-mon[32784]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:38:59.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:58 smithi094 ceph-mon[33032]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:00.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:59 smithi094 ceph-mon[33032]: from='client.? 172.21.15.94:0/774841171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50476b46-ec06-472f-bb91-920d81555099"}]: dispatch 2024-04-25T11:39:00.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:59 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50476b46-ec06-472f-bb91-920d81555099"}]: dispatch 2024-04-25T11:39:00.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:59 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50476b46-ec06-472f-bb91-920d81555099"}]': finished 2024-04-25T11:39:00.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:59 smithi094 ceph-mon[33032]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T11:39:00.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:38:59 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:00.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:59 smithi083 ceph-mon[29330]: from='client.? 172.21.15.94:0/774841171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50476b46-ec06-472f-bb91-920d81555099"}]: dispatch 2024-04-25T11:39:00.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:59 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50476b46-ec06-472f-bb91-920d81555099"}]: dispatch 2024-04-25T11:39:00.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:59 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50476b46-ec06-472f-bb91-920d81555099"}]': finished 2024-04-25T11:39:00.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:59 smithi083 ceph-mon[29330]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T11:39:00.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:38:59 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:00.238 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:59 smithi111 ceph-mon[32784]: from='client.? 172.21.15.94:0/774841171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50476b46-ec06-472f-bb91-920d81555099"}]: dispatch 2024-04-25T11:39:00.238 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:59 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50476b46-ec06-472f-bb91-920d81555099"}]: dispatch 2024-04-25T11:39:00.238 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:59 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50476b46-ec06-472f-bb91-920d81555099"}]': finished 2024-04-25T11:39:00.239 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:59 smithi111 ceph-mon[32784]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T11:39:00.239 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:38:59 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:01.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:00 smithi083 ceph-mon[29330]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:01.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:00 smithi083 ceph-mon[29330]: from='client.? 172.21.15.94:0/1911979282' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:39:01.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:00 smithi094 ceph-mon[33032]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:01.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:00 smithi094 ceph-mon[33032]: from='client.? 172.21.15.94:0/1911979282' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:39:01.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:00 smithi111 ceph-mon[32784]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:01.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:00 smithi111 ceph-mon[32784]: from='client.? 172.21.15.94:0/1911979282' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:39:03.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:02 smithi083 ceph-mon[29330]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:03.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:02 smithi094 ceph-mon[33032]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:03.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:02 smithi111 ceph-mon[32784]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:05.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:04 smithi083 ceph-mon[29330]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:05.246 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:04 smithi094 ceph-mon[33032]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:05.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:04 smithi111 ceph-mon[32784]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:07.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:06 smithi083 ceph-mon[29330]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:07.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T11:39:07.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:07.259 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:06 smithi094 ceph-mon[33032]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:07.259 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T11:39:07.259 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:07.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:06 smithi111 ceph-mon[32784]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:07.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T11:39:07.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:08.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:07 smithi083 ceph-mon[29330]: Deploying daemon osd.7 on smithi094 2024-04-25T11:39:08.233 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:07 smithi094 ceph-mon[33032]: Deploying daemon osd.7 on smithi094 2024-04-25T11:39:08.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:07 smithi111 ceph-mon[32784]: Deploying daemon osd.7 on smithi094 2024-04-25T11:39:09.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:08 smithi083 ceph-mon[29330]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:09.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:08 smithi094 ceph-mon[33032]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:09.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:08 smithi111 ceph-mon[32784]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:11.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:10 smithi083 ceph-mon[29330]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:11.247 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:10 smithi094 ceph-mon[33032]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:11.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:10 smithi111 ceph-mon[32784]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:13.060 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:12 smithi094 ceph-mon[33032]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:13.060 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:13.060 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:39:13.060 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:13.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:12 smithi083 ceph-mon[29330]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:13.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:13.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:39:13.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:13.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:12 smithi111 ceph-mon[32784]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:13.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:13.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:39:13.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:15.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:14 smithi083 ceph-mon[29330]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:15.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:14 smithi094 ceph-mon[33032]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:15.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:14 smithi111 ceph-mon[32784]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:15.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:39:15.687+0000 7f8504ff9700 1 -- 172.21.15.94:0/2548036342 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f850806b190 con 0x7f84f00a3c40 2024-04-25T11:39:15.675 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 7 on host 'smithi094' 2024-04-25T11:39:15.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:39:15.689+0000 7f850e092700 1 -- 172.21.15.94:0/2548036342 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84f00a3c40 msgr2=0x7f84f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:39:15.689+0000 7f850e092700 1 --2- 172.21.15.94:0/2548036342 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84f00a3c40 0x7f84f00a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f84fc006fd0 tx=0x7f84fc008040 comp rx=0 tx=0).stop 2024-04-25T11:39:15.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:39:15.689+0000 7f850e092700 1 -- 172.21.15.94:0/2548036342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85081019b0 msgr2=0x7f8508197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:15.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:39:15.689+0000 7f850e092700 1 --2- 172.21.15.94:0/2548036342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85081019b0 0x7f8508197c40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f84f8006010 tx=0x7f84f8005af0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:39:15.689+0000 7f850e092700 1 -- 172.21.15.94:0/2548036342 shutdown_connections 2024-04-25T11:39:15.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:39:15.689+0000 7f850e092700 1 --2- 172.21.15.94:0/2548036342 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84f00a3c40 0x7f84f00a6100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:39:15.689+0000 7f850e092700 1 --2- 172.21.15.94:0/2548036342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8508103550 0x7f850819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:39:15.689+0000 7f850e092700 1 --2- 172.21.15.94:0/2548036342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f85081019b0 0x7f8508197c40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:39:15.689+0000 7f850e092700 1 --2- 172.21.15.94:0/2548036342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8508102bb0 0x7f8508198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:15.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:39:15.689+0000 7f850e092700 1 -- 172.21.15.94:0/2548036342 >> 172.21.15.94:0/2548036342 conn(0x7f85080fcf70 msgr2=0x7f8508106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:15.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:39:15.689+0000 7f850e092700 1 -- 172.21.15.94:0/2548036342 shutdown_connections 2024-04-25T11:39:15.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:39:15.689+0000 7f850e092700 1 -- 172.21.15.94:0/2548036342 wait complete. 2024-04-25T11:39:16.364 DEBUG:teuthology.orchestra.run.smithi094:osd.7> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T11:39:16.367 INFO:tasks.cephadm:Deploying osd.8 on smithi111 with /dev/vg_nvme/lv_2... 2024-04-25T11:39:16.367 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T11:39:16.614 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:39:16.959 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:16 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:16.959 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:16 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:16.959 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:16 smithi111 ceph-mon[32784]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:16.959 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:16 smithi111 ceph-mon[32784]: from='osd.7 [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T11:39:16.959 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:16 smithi111 ceph-mon[32784]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T11:39:16.960 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:16 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:16.960 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:16 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:17.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:16 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:17.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:16 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:17.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:16 smithi094 ceph-mon[33032]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:17.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:16 smithi094 ceph-mon[33032]: from='osd.7 [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T11:39:17.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:16 smithi094 ceph-mon[33032]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T11:39:17.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:16 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:17.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:16 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:17.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:16 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:17.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:16 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:17.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:16 smithi083 ceph-mon[29330]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:17.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:16 smithi083 ceph-mon[29330]: from='osd.7 [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T11:39:17.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:16 smithi083 ceph-mon[29330]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T11:39:17.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:16 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:17.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:16 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:17.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:17 smithi094 ceph-mon[33032]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T11:39:17.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:17 smithi094 ceph-mon[33032]: from='osd.7 [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:39:17.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:17 smithi094 ceph-mon[33032]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T11:39:17.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:17 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:17.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:17 smithi094 ceph-mon[33032]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:39:17.901 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:17 smithi111 ceph-mon[32784]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T11:39:17.901 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:17 smithi111 ceph-mon[32784]: from='osd.7 [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:39:17.901 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:17 smithi111 ceph-mon[32784]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T11:39:17.901 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:17 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:17.901 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:17 smithi111 ceph-mon[32784]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:39:18.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:17 smithi083 ceph-mon[29330]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T11:39:18.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:17 smithi083 ceph-mon[29330]: from='osd.7 [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:39:18.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:17 smithi083 ceph-mon[29330]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T11:39:18.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:17 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:18.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:17 smithi083 ceph-mon[29330]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:39:18.448 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T11:39:18.449 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T11:39:18.449 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-25T11:39:18.449 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-25T11:39:18.449 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112597 s, 931 MB/s 2024-04-25T11:39:18.449 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-25T11:39:18.480 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:39:18 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:39:18.379+0000 7ff213b60700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T11:39:18.480 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:39:18 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:39:18.389+0000 7ff20b94b700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:39:18.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:18 smithi094 ceph-mon[33032]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T11:39:18.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:18 smithi094 ceph-mon[33032]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T11:39:18.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:18.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:18.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:18 smithi094 ceph-mon[33032]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:18.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:18 smithi094 ceph-mon[33032]: from='osd.7 ' entity='osd.7' 2024-04-25T11:39:19.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:18 smithi111 ceph-mon[32784]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T11:39:19.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:18 smithi111 ceph-mon[32784]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T11:39:19.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:19.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:19.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:18 smithi111 ceph-mon[32784]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:19.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:18 smithi111 ceph-mon[32784]: from='osd.7 ' entity='osd.7' 2024-04-25T11:39:19.123 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_2 2024-04-25T11:39:19.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:18 smithi083 ceph-mon[29330]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T11:39:19.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:18 smithi083 ceph-mon[29330]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T11:39:19.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:19.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:19.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:18 smithi083 ceph-mon[29330]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-25T11:39:19.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:18 smithi083 ceph-mon[29330]: from='osd.7 ' entity='osd.7' 2024-04-25T11:39:19.366 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:39:19.763 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:19 smithi111 ceph-mon[32784]: purged_snaps scrub starts 2024-04-25T11:39:19.763 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:19 smithi111 ceph-mon[32784]: purged_snaps scrub ok 2024-04-25T11:39:19.763 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:19 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:19.763 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:19 smithi111 ceph-mon[32784]: osd.7 [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463] boot 2024-04-25T11:39:19.763 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:19 smithi111 ceph-mon[32784]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T11:39:19.763 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:19 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:19.882 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:19 smithi094 ceph-mon[33032]: purged_snaps scrub starts 2024-04-25T11:39:19.882 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:19 smithi094 ceph-mon[33032]: purged_snaps scrub ok 2024-04-25T11:39:19.882 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:19 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:19.882 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:19 smithi094 ceph-mon[33032]: osd.7 [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463] boot 2024-04-25T11:39:19.883 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:19 smithi094 ceph-mon[33032]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T11:39:19.883 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:19 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:20.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:19 smithi083 ceph-mon[29330]: purged_snaps scrub starts 2024-04-25T11:39:20.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:19 smithi083 ceph-mon[29330]: purged_snaps scrub ok 2024-04-25T11:39:20.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:19 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:20.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:19 smithi083 ceph-mon[29330]: osd.7 [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463] boot 2024-04-25T11:39:20.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:19 smithi083 ceph-mon[29330]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T11:39:20.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:19 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:39:20.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.869+0000 7f8f5c2d0700 1 -- 172.21.15.111:0/2980485182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f54103550 msgr2=0x7f8f54109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.869+0000 7f8f5c2d0700 1 --2- 172.21.15.111:0/2980485182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f54103550 0x7f8f54109e00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00b0a0 tx=0x7f8f4c00b3b0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.870+0000 7f8f5c2d0700 1 -- 172.21.15.111:0/2980485182 shutdown_connections 2024-04-25T11:39:20.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.870+0000 7f8f5c2d0700 1 --2- 172.21.15.111:0/2980485182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f54103550 0x7f8f54109e00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.870+0000 7f8f5c2d0700 1 --2- 172.21.15.111:0/2980485182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f541019b0 0x7f8f54101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.870+0000 7f8f5c2d0700 1 --2- 172.21.15.111:0/2980485182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f54102bb0 0x7f8f54103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.870+0000 7f8f5c2d0700 1 -- 172.21.15.111:0/2980485182 >> 172.21.15.111:0/2980485182 conn(0x7f8f540fcf70 msgr2=0x7f8f540ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:20.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.870+0000 7f8f5c2d0700 1 -- 172.21.15.111:0/2980485182 shutdown_connections 2024-04-25T11:39:20.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.870+0000 7f8f5c2d0700 1 -- 172.21.15.111:0/2980485182 wait complete. 2024-04-25T11:39:20.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.870+0000 7f8f5c2d0700 1 Processor -- start 2024-04-25T11:39:20.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.870+0000 7f8f5c2d0700 1 -- start start 2024-04-25T11:39:20.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.870+0000 7f8f5c2d0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f541019b0 0x7f8f54198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.870+0000 7f8f5c2d0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f54102bb0 0x7f8f54199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.870+0000 7f8f5c2d0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f54103550 0x7f8f541a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.870+0000 7f8f5c2d0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5410d450 con 0x7f8f54103550 2024-04-25T11:39:20.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.870+0000 7f8f5c2d0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5410d2d0 con 0x7f8f54102bb0 2024-04-25T11:39:20.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.870+0000 7f8f5c2d0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5410d5d0 con 0x7f8f541019b0 2024-04-25T11:39:20.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f5a06c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f541019b0 0x7f8f54198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f5a06c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f541019b0 0x7f8f54198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60756/0 (socket says 172.21.15.111:60756) 2024-04-25T11:39:20.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f5a06c700 1 -- 172.21.15.111:0/3573751530 learned_addr learned my addr 172.21.15.111:0/3573751530 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-25T11:39:20.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f5a86d700 1 --2- 172.21.15.111:0/3573751530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f54103550 0x7f8f541a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f5986b700 1 --2- 172.21.15.111:0/3573751530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f54102bb0 0x7f8f54199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f5a06c700 1 -- 172.21.15.111:0/3573751530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f54102bb0 msgr2=0x7f8f54199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f5a06c700 1 --2- 172.21.15.111:0/3573751530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f54102bb0 0x7f8f54199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f5a06c700 1 -- 172.21.15.111:0/3573751530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f54103550 msgr2=0x7f8f541a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:20.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f5a06c700 1 --2- 172.21.15.111:0/3573751530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f54103550 0x7f8f541a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:20.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f5a06c700 1 -- 172.21.15.111:0/3573751530 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f541a0830 con 0x7f8f541019b0 2024-04-25T11:39:20.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f5a06c700 1 --2- 172.21.15.111:0/3573751530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f541019b0 0x7f8f54198c20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000d9b0 tx=0x7f8f5000dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:20.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f4b7fe700 1 -- 172.21.15.111:0/3573751530 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50016070 con 0x7f8f541019b0 2024-04-25T11:39:20.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f5c2d0700 1 -- 172.21.15.111:0/3573751530 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f541a0b20 con 0x7f8f541019b0 2024-04-25T11:39:20.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f4b7fe700 1 -- 172.21.15.111:0/3573751530 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f5000bb60 con 0x7f8f541019b0 2024-04-25T11:39:20.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f4b7fe700 1 -- 172.21.15.111:0/3573751530 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5000f3c0 con 0x7f8f541019b0 2024-04-25T11:39:20.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.871+0000 7f8f5c2d0700 1 -- 172.21.15.111:0/3573751530 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f541a1060 con 0x7f8f541019b0 2024-04-25T11:39:20.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.872+0000 7f8f5c2d0700 1 -- 172.21.15.111:0/3573751530 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f54069730 con 0x7f8f541019b0 2024-04-25T11:39:20.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.874+0000 7f8f4b7fe700 1 -- 172.21.15.111:0/3573751530 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8f50019070 con 0x7f8f541019b0 2024-04-25T11:39:20.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.874+0000 7f8f4b7fe700 1 --2- 172.21.15.111:0/3573751530 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8f400a3c40 0x7f8f400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:20.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.874+0000 7f8f4b7fe700 1 -- 172.21.15.111:0/3573751530 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f8f500cfe50 con 0x7f8f541019b0 2024-04-25T11:39:20.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.875+0000 7f8f5986b700 1 --2- 172.21.15.111:0/3573751530 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8f400a3c40 0x7f8f400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:20.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.875+0000 7f8f5986b700 1 --2- 172.21.15.111:0/3573751530 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8f400a3c40 0x7f8f400a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8f44005fd0 tx=0x7f8f44005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:20.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:20.878+0000 7f8f4b7fe700 1 -- 172.21.15.111:0/3573751530 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f50020030 con 0x7f8f541019b0 2024-04-25T11:39:21.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T11:39:21.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: pgmap v178: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:21.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: Detected new or changed devices on smithi094 2024-04-25T11:39:21.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:21.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:21.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:21.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:21.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:21.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi094 to 5696M 2024-04-25T11:39:21.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:21.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:21.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:39:21.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:21.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:39:21.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:21.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:39:21.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:21.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T11:39:21.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: pgmap v178: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:21.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: Detected new or changed devices on smithi094 2024-04-25T11:39:21.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:21.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:21.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:21.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:21.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:21.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi094 to 5696M 2024-04-25T11:39:21.057 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:21.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:21.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:39:21.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:21.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:39:21.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:21.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:39:21.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:21.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:21.084+0000 7f8f5c2d0700 1 -- 172.21.15.111:0/3573751530 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8f5406b8a0 con 0x7f8f400a3c40 2024-04-25T11:39:21.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T11:39:21.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: pgmap v178: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:21.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: Detected new or changed devices on smithi094 2024-04-25T11:39:21.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:21.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:21.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:21.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:21.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:21.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi094 to 5696M 2024-04-25T11:39:21.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:21.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:21.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:39:21.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:21.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:39:21.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:21.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:39:21.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:21.989 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:21 smithi111 ceph-mon[32784]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T11:39:21.989 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:21 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:39:21.989 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:21 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:39:21.989 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:21 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:21.989 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:21 smithi111 ceph-mon[32784]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T11:39:22.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:21 smithi094 ceph-mon[33032]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T11:39:22.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:21 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:39:22.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:21 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:39:22.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:21 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:22.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:21 smithi094 ceph-mon[33032]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T11:39:22.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:21 smithi083 ceph-mon[29330]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T11:39:22.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:21 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:39:22.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:21 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:39:22.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:21 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:22.073 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:21 smithi083 ceph-mon[29330]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T11:39:22.978 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:22 smithi111 ceph-mon[32784]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:39:22.979 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:22 smithi111 ceph-mon[32784]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:22.979 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:22 smithi111 ceph-mon[32784]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T11:39:23.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:22 smithi094 ceph-mon[33032]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:39:23.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:22 smithi094 ceph-mon[33032]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:23.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:22 smithi094 ceph-mon[33032]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T11:39:23.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:22 smithi083 ceph-mon[29330]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:39:23.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:22 smithi083 ceph-mon[29330]: pgmap v181: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:23.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:22 smithi083 ceph-mon[29330]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T11:39:24.124 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:23 smithi111 ceph-mon[32784]: from='client.? 172.21.15.111:0/3979875985' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34ebcd68-bca8-4d62-b0df-728fd892a0b5"}]: dispatch 2024-04-25T11:39:24.124 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:23 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34ebcd68-bca8-4d62-b0df-728fd892a0b5"}]: dispatch 2024-04-25T11:39:24.124 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:23 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34ebcd68-bca8-4d62-b0df-728fd892a0b5"}]': finished 2024-04-25T11:39:24.124 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:23 smithi111 ceph-mon[32784]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T11:39:24.124 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:23 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:39:24.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:23 smithi083 ceph-mon[29330]: from='client.? 172.21.15.111:0/3979875985' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34ebcd68-bca8-4d62-b0df-728fd892a0b5"}]: dispatch 2024-04-25T11:39:24.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:23 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34ebcd68-bca8-4d62-b0df-728fd892a0b5"}]: dispatch 2024-04-25T11:39:24.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:23 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34ebcd68-bca8-4d62-b0df-728fd892a0b5"}]': finished 2024-04-25T11:39:24.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:23 smithi083 ceph-mon[29330]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T11:39:24.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:23 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:39:24.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:23 smithi094 ceph-mon[33032]: from='client.? 172.21.15.111:0/3979875985' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34ebcd68-bca8-4d62-b0df-728fd892a0b5"}]: dispatch 2024-04-25T11:39:24.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:23 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "34ebcd68-bca8-4d62-b0df-728fd892a0b5"}]: dispatch 2024-04-25T11:39:24.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:23 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "34ebcd68-bca8-4d62-b0df-728fd892a0b5"}]': finished 2024-04-25T11:39:24.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:23 smithi094 ceph-mon[33032]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T11:39:24.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:23 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:39:25.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:24 smithi083 ceph-mon[29330]: pgmap v184: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:25.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:24 smithi083 ceph-mon[29330]: from='client.? 172.21.15.111:0/2115563677' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:39:25.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:24 smithi094 ceph-mon[33032]: pgmap v184: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:25.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:24 smithi094 ceph-mon[33032]: from='client.? 172.21.15.111:0/2115563677' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:39:25.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:24 smithi111 ceph-mon[32784]: pgmap v184: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:25.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:24 smithi111 ceph-mon[32784]: from='client.? 172.21.15.111:0/2115563677' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:39:27.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:26 smithi094 ceph-mon[33032]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:27.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:26 smithi111 ceph-mon[32784]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:26 smithi083 ceph-mon[29330]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:29.162 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:28 smithi111 ceph-mon[32784]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:29.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:28 smithi094 ceph-mon[33032]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:29.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:28 smithi083 ceph-mon[29330]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:31.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:30 smithi094 ceph-mon[33032]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:31.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:30 smithi111 ceph-mon[32784]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:31.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:30 smithi083 ceph-mon[29330]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:32.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:31 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T11:39:32.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:31 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:32.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:31 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T11:39:32.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:31 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:32.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:31 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T11:39:32.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:31 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:33.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:32 smithi111 ceph-mon[32784]: Deploying daemon osd.8 on smithi111 2024-04-25T11:39:33.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:32 smithi111 ceph-mon[32784]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:33.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:32 smithi083 ceph-mon[29330]: Deploying daemon osd.8 on smithi111 2024-04-25T11:39:33.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:32 smithi083 ceph-mon[29330]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:33.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:32 smithi094 ceph-mon[33032]: Deploying daemon osd.8 on smithi111 2024-04-25T11:39:33.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:32 smithi094 ceph-mon[33032]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:35.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:34 smithi094 ceph-mon[33032]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:35.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:34 smithi111 ceph-mon[32784]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:35.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:34 smithi083 ceph-mon[29330]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:37.233 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:36 smithi111 ceph-mon[32784]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:37.234 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:36 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:37.234 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:36 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:37.234 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:36 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:39:37.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:36 smithi094 ceph-mon[33032]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:37.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:36 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:37.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:36 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:37.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:36 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:39:37.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:36 smithi083 ceph-mon[29330]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:37.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:36 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:37.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:36 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:37.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:36 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:39:39.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:38 smithi094 ceph-mon[33032]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:39.309 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:38 smithi111 ceph-mon[32784]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:39.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:38 smithi083 ceph-mon[29330]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:39.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:39.949+0000 7f8f4b7fe700 1 -- 172.21.15.111:0/3573751530 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8f5406b8a0 con 0x7f8f400a3c40 2024-04-25T11:39:39.937 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 8 on host 'smithi111' 2024-04-25T11:39:39.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:39.952+0000 7f8f5c2d0700 1 -- 172.21.15.111:0/3573751530 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8f400a3c40 msgr2=0x7f8f400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:39.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:39.952+0000 7f8f5c2d0700 1 --2- 172.21.15.111:0/3573751530 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8f400a3c40 0x7f8f400a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8f44005fd0 tx=0x7f8f44005ee0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:39.952+0000 7f8f5c2d0700 1 -- 172.21.15.111:0/3573751530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f541019b0 msgr2=0x7f8f54198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:39.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:39.952+0000 7f8f5c2d0700 1 --2- 172.21.15.111:0/3573751530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f541019b0 0x7f8f54198c20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000d9b0 tx=0x7f8f5000dd70 comp rx=0 tx=0).stop 2024-04-25T11:39:39.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:39.952+0000 7f8f5c2d0700 1 -- 172.21.15.111:0/3573751530 shutdown_connections 2024-04-25T11:39:39.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:39.952+0000 7f8f5c2d0700 1 --2- 172.21.15.111:0/3573751530 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8f400a3c40 0x7f8f400a6100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:39.952+0000 7f8f5c2d0700 1 --2- 172.21.15.111:0/3573751530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f54103550 0x7f8f541a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.939 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:39.952+0000 7f8f5c2d0700 1 --2- 172.21.15.111:0/3573751530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f541019b0 0x7f8f54198c20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:39.952+0000 7f8f5c2d0700 1 --2- 172.21.15.111:0/3573751530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f54102bb0 0x7f8f54199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:39.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:39.952+0000 7f8f5c2d0700 1 -- 172.21.15.111:0/3573751530 >> 172.21.15.111:0/3573751530 conn(0x7f8f540fcf70 msgr2=0x7f8f5406b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:39.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:39.952+0000 7f8f5c2d0700 1 -- 172.21.15.111:0/3573751530 shutdown_connections 2024-04-25T11:39:39.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:39:39.952+0000 7f8f5c2d0700 1 -- 172.21.15.111:0/3573751530 wait complete. 2024-04-25T11:39:40.551 DEBUG:teuthology.orchestra.run.smithi111:osd.8> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T11:39:40.554 INFO:tasks.cephadm:Deploying osd.9 on smithi083 with /dev/vg_nvme/lv_1... 2024-04-25T11:39:40.554 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T11:39:40.818 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:39:41.151 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:40 smithi083 ceph-mon[29330]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:41.151 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:41.151 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:41.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:40 smithi083 ceph-mon[29330]: from='osd.8 [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T11:39:41.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:40 smithi083 ceph-mon[29330]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T11:39:41.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:41.152 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:41.272 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:40 smithi111 ceph-mon[32784]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:41.272 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:41.273 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:41.273 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:40 smithi111 ceph-mon[32784]: from='osd.8 [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T11:39:41.273 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:40 smithi111 ceph-mon[32784]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T11:39:41.273 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:41.273 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:40 smithi094 ceph-mon[33032]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:40 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:40 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:40 smithi094 ceph-mon[33032]: from='osd.8 [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T11:39:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:40 smithi094 ceph-mon[33032]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T11:39:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:40 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:41.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:40 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:42.173 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:41 smithi111 ceph-mon[32784]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T11:39:42.173 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:41 smithi111 ceph-mon[32784]: from='osd.8 [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:39:42.173 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:41 smithi111 ceph-mon[32784]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T11:39:42.173 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:41 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:39:42.173 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:41 smithi111 ceph-mon[32784]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:39:42.174 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:39:41 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:39:41.982+0000 7f5698893700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T11:39:42.174 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:39:41 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:39:41.996+0000 7f5694686700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:39:42.209 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:41 smithi083 ceph-mon[29330]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T11:39:42.209 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:41 smithi083 ceph-mon[29330]: from='osd.8 [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:39:42.209 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:41 smithi083 ceph-mon[29330]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T11:39:42.209 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:41 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:39:42.209 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:41 smithi083 ceph-mon[29330]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:39:42.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:41 smithi094 ceph-mon[33032]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T11:39:42.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:41 smithi094 ceph-mon[33032]: from='osd.8 [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:39:42.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:41 smithi094 ceph-mon[33032]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T11:39:42.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:41 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:39:42.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:41 smithi094 ceph-mon[33032]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:39:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T11:39:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T11:39:42.747 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-25T11:39:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-25T11:39:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115849 s, 905 MB/s 2024-04-25T11:39:42.747 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-25T11:39:43.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:42 smithi111 ceph-mon[32784]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:43.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:42 smithi111 ceph-mon[32784]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-25T11:39:43.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:42 smithi111 ceph-mon[32784]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T11:39:43.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:42 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:39:43.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:42 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:39:43.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:42 smithi094 ceph-mon[33032]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:43.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:42 smithi094 ceph-mon[33032]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-25T11:39:43.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:42 smithi094 ceph-mon[33032]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T11:39:43.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:42 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:39:43.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:42 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:39:43.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:42 smithi083 ceph-mon[29330]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T11:39:43.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:42 smithi083 ceph-mon[29330]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-25T11:39:43.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:42 smithi083 ceph-mon[29330]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T11:39:43.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:42 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:39:43.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:42 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:39:43.475 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_1 2024-04-25T11:39:43.722 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:39:44.013 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:43 smithi083 ceph-mon[29330]: purged_snaps scrub starts 2024-04-25T11:39:44.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:43 smithi083 ceph-mon[29330]: purged_snaps scrub ok 2024-04-25T11:39:44.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:43 smithi083 ceph-mon[29330]: osd.8 [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032] boot 2024-04-25T11:39:44.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:43 smithi083 ceph-mon[29330]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T11:39:44.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:43 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:39:44.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:43 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:44.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:43 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:44.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:43 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:44.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:43 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:44.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:43 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:44.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:43 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:44.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:43 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:44.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:43 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:39:44.015 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:43 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:44.015 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:43 smithi083 ceph-mon[29330]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T11:39:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:43 smithi094 ceph-mon[33032]: purged_snaps scrub starts 2024-04-25T11:39:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:43 smithi094 ceph-mon[33032]: purged_snaps scrub ok 2024-04-25T11:39:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:43 smithi094 ceph-mon[33032]: osd.8 [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032] boot 2024-04-25T11:39:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:43 smithi094 ceph-mon[33032]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T11:39:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:43 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:39:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:43 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:43 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:43 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:43 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:43 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:44.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:43 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:44.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:43 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:44.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:43 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:39:44.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:43 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:44.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:43 smithi094 ceph-mon[33032]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T11:39:44.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:43 smithi111 ceph-mon[32784]: purged_snaps scrub starts 2024-04-25T11:39:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:43 smithi111 ceph-mon[32784]: purged_snaps scrub ok 2024-04-25T11:39:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:43 smithi111 ceph-mon[32784]: osd.8 [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032] boot 2024-04-25T11:39:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:43 smithi111 ceph-mon[32784]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T11:39:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:43 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:39:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:43 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:43 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:43 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:43 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:43 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:39:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:43 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:43 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:43 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:39:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:43 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:44.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:43 smithi111 ceph-mon[32784]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T11:39:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.233+0000 7f68593c6700 1 -- 172.21.15.83:0/1923749570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6854102620 msgr2=0x7f6854109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.233+0000 7f68593c6700 1 --2- 172.21.15.83:0/1923749570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6854102620 0x7f6854109ee0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f684c00b0a0 tx=0x7f684c00b3b0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.233+0000 7f68593c6700 1 -- 172.21.15.83:0/1923749570 shutdown_connections 2024-04-25T11:39:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.233+0000 7f68593c6700 1 --2- 172.21.15.83:0/1923749570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6854102620 0x7f6854109ee0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.233+0000 7f68593c6700 1 --2- 172.21.15.83:0/1923749570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6854102fd0 0x7f68541033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.233+0000 7f68593c6700 1 --2- 172.21.15.83:0/1923749570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6854101c80 0x7f68541020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.233+0000 7f68593c6700 1 -- 172.21.15.83:0/1923749570 >> 172.21.15.83:0/1923749570 conn(0x7f68540fc710 msgr2=0x7f68540feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:39:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.233+0000 7f68593c6700 1 -- 172.21.15.83:0/1923749570 shutdown_connections 2024-04-25T11:39:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.233+0000 7f68593c6700 1 -- 172.21.15.83:0/1923749570 wait complete. 2024-04-25T11:39:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68593c6700 1 Processor -- start 2024-04-25T11:39:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68593c6700 1 -- start start 2024-04-25T11:39:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68593c6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6854101c80 0x7f6854101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68593c6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6854102620 0x7f685419d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68593c6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6854102fd0 0x7f685419fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68593c6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f685410d460 con 0x7f6854102fd0 2024-04-25T11:39:45.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68593c6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f685410d2e0 con 0x7f6854102620 2024-04-25T11:39:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68593c6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f685410d5e0 con 0x7f6854101c80 2024-04-25T11:39:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68537fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6854102fd0 0x7f685419fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68537fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6854102fd0 0x7f685419fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59182/0 (socket says 172.21.15.83:59182) 2024-04-25T11:39:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68537fe700 1 -- 172.21.15.83:0/2948858773 learned_addr learned my addr 172.21.15.83:0/2948858773 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:39:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f6852ffd700 1 --2- 172.21.15.83:0/2948858773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6854101c80 0x7f6854101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f684a7fc700 1 --2- 172.21.15.83:0/2948858773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6854102620 0x7f685419d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68537fe700 1 -- 172.21.15.83:0/2948858773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6854101c80 msgr2=0x7f6854101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68537fe700 1 --2- 172.21.15.83:0/2948858773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6854101c80 0x7f6854101730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68537fe700 1 -- 172.21.15.83:0/2948858773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6854102620 msgr2=0x7f685419d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:39:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68537fe700 1 --2- 172.21.15.83:0/2948858773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6854102620 0x7f685419d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:39:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f68537fe700 1 -- 172.21.15.83:0/2948858773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68541a0770 con 0x7f6854102fd0 2024-04-25T11:39:45.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.234+0000 7f684a7fc700 1 --2- 172.21.15.83:0/2948858773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6854102620 0x7f685419d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:39:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.235+0000 7f68537fe700 1 --2- 172.21.15.83:0/2948858773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6854102fd0 0x7f685419fea0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f684c0062a0 tx=0x7f684c004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.235+0000 7f6850ff9700 1 -- 172.21.15.83:0/2948858773 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c021070 con 0x7f6854102fd0 2024-04-25T11:39:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.235+0000 7f6850ff9700 1 -- 172.21.15.83:0/2948858773 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f684c007c80 con 0x7f6854102fd0 2024-04-25T11:39:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.235+0000 7f68593c6700 1 -- 172.21.15.83:0/2948858773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68541a0a00 con 0x7f6854102fd0 2024-04-25T11:39:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.235+0000 7f6850ff9700 1 -- 172.21.15.83:0/2948858773 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c004850 con 0x7f6854102fd0 2024-04-25T11:39:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.235+0000 7f68593c6700 1 -- 172.21.15.83:0/2948858773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68541a0f90 con 0x7f6854102fd0 2024-04-25T11:39:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.236+0000 7f68593c6700 1 -- 172.21.15.83:0/2948858773 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6854069730 con 0x7f6854102fd0 2024-04-25T11:39:45.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.237+0000 7f6850ff9700 1 -- 172.21.15.83:0/2948858773 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f684c003a00 con 0x7f6854102fd0 2024-04-25T11:39:45.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.238+0000 7f6850ff9700 1 --2- 172.21.15.83:0/2948858773 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f68400a3cc0 0x7f68400a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:39:45.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.238+0000 7f6852ffd700 1 --2- 172.21.15.83:0/2948858773 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f68400a3cc0 0x7f68400a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:39:45.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.238+0000 7f6850ff9700 1 -- 172.21.15.83:0/2948858773 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7f684c00f020 con 0x7f6854102fd0 2024-04-25T11:39:45.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.238+0000 7f6852ffd700 1 --2- 172.21.15.83:0/2948858773 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f68400a3cc0 0x7f68400a6180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f685406af30 tx=0x7f684400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:39:45.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.241+0000 7f6850ff9700 1 -- 172.21.15.83:0/2948858773 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f684c02a020 con 0x7f6854102fd0 2024-04-25T11:39:45.229 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:44 smithi083 ceph-mon[29330]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:39:45.229 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:44 smithi083 ceph-mon[29330]: Detected new or changed devices on smithi111 2024-04-25T11:39:45.229 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:44 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi111 to 5696M 2024-04-25T11:39:45.229 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:44 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:39:45.229 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:44 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:45.229 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:44 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:39:45.229 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:44 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:45.229 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:44 smithi083 ceph-mon[29330]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T11:39:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:44 smithi094 ceph-mon[33032]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:39:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:44 smithi094 ceph-mon[33032]: Detected new or changed devices on smithi111 2024-04-25T11:39:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:44 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi111 to 5696M 2024-04-25T11:39:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:44 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:39:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:44 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:44 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:39:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:44 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:44 smithi094 ceph-mon[33032]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T11:39:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:44 smithi111 ceph-mon[32784]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:39:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:44 smithi111 ceph-mon[32784]: Detected new or changed devices on smithi111 2024-04-25T11:39:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:44 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi111 to 5696M 2024-04-25T11:39:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:44 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:39:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:44 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:44 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:39:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:44 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:39:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:44 smithi111 ceph-mon[32784]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T11:39:45.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:39:45.448+0000 7f68593c6700 1 -- 172.21.15.83:0/2948858773 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6854062f40 con 0x7f68400a3cc0 2024-04-25T11:39:46.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:45 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:39:46.260 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:45 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:39:46.260 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:45 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:46.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:45 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:39:46.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:45 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:39:46.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:45 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:46.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:45 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:39:46.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:45 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:39:46.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:45 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:47.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:47 smithi094 ceph-mon[33032]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:39:47.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:47 smithi094 ceph-mon[33032]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:39:47.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:47 smithi094 ceph-mon[33032]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T11:39:47.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:47 smithi111 ceph-mon[32784]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:39:47.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:47 smithi111 ceph-mon[32784]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:39:47.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:47 smithi111 ceph-mon[32784]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T11:39:47.327 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:47 smithi083 ceph-mon[29330]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:39:47.327 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:47 smithi083 ceph-mon[29330]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:39:47.327 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:47 smithi083 ceph-mon[29330]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T11:39:48.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:48 smithi094 ceph-mon[33032]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T11:39:48.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:48 smithi111 ceph-mon[32784]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T11:39:48.400 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:48 smithi083 ceph-mon[29330]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T11:39:49.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:49 smithi094 ceph-mon[33032]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:39:49.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:49 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3693220135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f70803a-8d30-40cd-8374-fefeb1bfc337"}]: dispatch 2024-04-25T11:39:49.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:49 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3693220135' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f70803a-8d30-40cd-8374-fefeb1bfc337"}]': finished 2024-04-25T11:39:49.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:49 smithi094 ceph-mon[33032]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T11:39:49.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:49 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:39:49.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:49 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3591359487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:39:49.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:49 smithi111 ceph-mon[32784]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:39:49.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:49 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3693220135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f70803a-8d30-40cd-8374-fefeb1bfc337"}]: dispatch 2024-04-25T11:39:49.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:49 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3693220135' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f70803a-8d30-40cd-8374-fefeb1bfc337"}]': finished 2024-04-25T11:39:49.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:49 smithi111 ceph-mon[32784]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T11:39:49.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:49 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:39:49.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:49 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3591359487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:39:49.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:49 smithi083 ceph-mon[29330]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:39:49.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:49 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3693220135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f70803a-8d30-40cd-8374-fefeb1bfc337"}]: dispatch 2024-04-25T11:39:49.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:49 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3693220135' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f70803a-8d30-40cd-8374-fefeb1bfc337"}]': finished 2024-04-25T11:39:49.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:49 smithi083 ceph-mon[29330]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T11:39:49.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:49 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:39:49.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:49 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3591359487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:39:51.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:51 smithi111 ceph-mon[32784]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-25T11:39:51.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:51 smithi083 ceph-mon[29330]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-25T11:39:51.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:51 smithi094 ceph-mon[33032]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-25T11:39:53.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:53 smithi111 ceph-mon[32784]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T11:39:53.399 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:53 smithi083 ceph-mon[29330]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T11:39:53.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:53 smithi094 ceph-mon[33032]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T11:39:55.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:55 smithi111 ceph-mon[32784]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T11:39:55.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:55 smithi083 ceph-mon[29330]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T11:39:55.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:55 smithi094 ceph-mon[33032]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T11:39:56.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:56 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T11:39:56.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:56 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:56.362 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:56 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T11:39:56.362 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:56 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:56.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:56 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T11:39:56.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:56 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:39:57.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:57 smithi111 ceph-mon[32784]: Deploying daemon osd.9 on smithi083 2024-04-25T11:39:57.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:57 smithi111 ceph-mon[32784]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-25T11:39:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:57 smithi083 ceph-mon[29330]: Deploying daemon osd.9 on smithi083 2024-04-25T11:39:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:57 smithi083 ceph-mon[29330]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-25T11:39:57.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:57 smithi094 ceph-mon[33032]: Deploying daemon osd.9 on smithi083 2024-04-25T11:39:57.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:57 smithi094 ceph-mon[33032]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-25T11:39:59.294 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:39:59 smithi083 ceph-mon[29330]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:39:59.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:39:59 smithi111 ceph-mon[32784]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:39:59.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:39:59 smithi094 ceph-mon[33032]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:40:00.200 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:00 smithi083 ceph-mon[29330]: overall HEALTH_OK 2024-04-25T11:40:00.316 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:00 smithi111 ceph-mon[32784]: overall HEALTH_OK 2024-04-25T11:40:00.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:00 smithi094 ceph-mon[33032]: overall HEALTH_OK 2024-04-25T11:40:01.251 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:01 smithi083 ceph-mon[29330]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-25T11:40:01.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:01 smithi111 ceph-mon[32784]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-25T11:40:01.424 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:01 smithi094 ceph-mon[33032]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-25T11:40:02.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:02 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:02.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:02 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:02.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:02 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:02.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:02 smithi094 ceph-mon[33032]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:40:02.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:02 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:02.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:02 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:02.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:02 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:02.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:02 smithi111 ceph-mon[32784]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:40:02.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:02 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:02.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:02 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:02.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:02 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:02.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:02 smithi083 ceph-mon[29330]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:40:05.014 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:04 smithi083 ceph-mon[29330]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:40:05.092 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 9 on host 'smithi083' 2024-04-25T11:40:05.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:05.105+0000 7f6850ff9700 1 -- 172.21.15.83:0/2948858773 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6854062f40 con 0x7f68400a3cc0 2024-04-25T11:40:05.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:05.108+0000 7f68593c6700 1 -- 172.21.15.83:0/2948858773 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f68400a3cc0 msgr2=0x7f68400a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:05.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:05.108+0000 7f68593c6700 1 --2- 172.21.15.83:0/2948858773 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f68400a3cc0 0x7f68400a6180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f685406af30 tx=0x7f684400a3b0 comp rx=0 tx=0).stop 2024-04-25T11:40:05.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:05.108+0000 7f68593c6700 1 -- 172.21.15.83:0/2948858773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6854102fd0 msgr2=0x7f685419fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:05.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:05.108+0000 7f68593c6700 1 --2- 172.21.15.83:0/2948858773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6854102fd0 0x7f685419fea0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f684c0062a0 tx=0x7f684c004e40 comp rx=0 tx=0).stop 2024-04-25T11:40:05.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:05.108+0000 7f68593c6700 1 -- 172.21.15.83:0/2948858773 shutdown_connections 2024-04-25T11:40:05.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:05.108+0000 7f68593c6700 1 --2- 172.21.15.83:0/2948858773 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f68400a3cc0 0x7f68400a6180 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:05.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:05.108+0000 7f68593c6700 1 --2- 172.21.15.83:0/2948858773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6854102fd0 0x7f685419fea0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:05.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:05.108+0000 7f68593c6700 1 --2- 172.21.15.83:0/2948858773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6854101c80 0x7f6854101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:05.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:05.108+0000 7f68593c6700 1 --2- 172.21.15.83:0/2948858773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6854102620 0x7f685419d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:05.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:05.108+0000 7f68593c6700 1 -- 172.21.15.83:0/2948858773 >> 172.21.15.83:0/2948858773 conn(0x7f68540fc710 msgr2=0x7f6854105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:05.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:05.108+0000 7f68593c6700 1 -- 172.21.15.83:0/2948858773 shutdown_connections 2024-04-25T11:40:05.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:05.108+0000 7f68593c6700 1 -- 172.21.15.83:0/2948858773 wait complete. 2024-04-25T11:40:05.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:04 smithi094 ceph-mon[33032]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:40:05.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:04 smithi111 ceph-mon[32784]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:40:05.804 DEBUG:teuthology.orchestra.run.smithi083:osd.9> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T11:40:05.807 INFO:tasks.cephadm:Deploying osd.10 on smithi094 with /dev/vg_nvme/lv_1... 2024-04-25T11:40:05.808 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T11:40:06.061 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:40:06.183 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:06.183 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:06.183 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:06 smithi094 ceph-mon[33032]: from='osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T11:40:06.183 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:06.183 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:06 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:06.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:06.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:06.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:06 smithi111 ceph-mon[32784]: from='osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T11:40:06.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:06.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:06 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:06.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:06.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:06.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:06 smithi083 ceph-mon[29330]: from='osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T11:40:06.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:06.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:06 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:07.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:07 smithi111 ceph-mon[32784]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:40:07.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:07 smithi111 ceph-mon[32784]: from='osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T11:40:07.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:07 smithi111 ceph-mon[32784]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T11:40:07.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:07 smithi111 ceph-mon[32784]: from='osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T11:40:07.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:07 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:40:07.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:07 smithi094 ceph-mon[33032]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:40:07.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:07 smithi094 ceph-mon[33032]: from='osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T11:40:07.382 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:07 smithi094 ceph-mon[33032]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T11:40:07.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:07 smithi094 ceph-mon[33032]: from='osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T11:40:07.383 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:07 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:40:07.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:07 smithi083 ceph-mon[29330]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:40:07.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:07 smithi083 ceph-mon[29330]: from='osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T11:40:07.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:07 smithi083 ceph-mon[29330]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T11:40:07.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:07 smithi083 ceph-mon[29330]: from='osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-25T11:40:07.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:07 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:40:08.004 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:40:07 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:40:07.985+0000 7fd321164700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T11:40:08.046 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T11:40:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T11:40:08.047 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-25T11:40:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-25T11:40:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00947697 s, 1.1 GB/s 2024-04-25T11:40:08.047 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-25T11:40:08.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:08 smithi111 ceph-mon[32784]: from='osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T11:40:08.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:08 smithi111 ceph-mon[32784]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T11:40:08.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:08 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:40:08.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:08 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:40:08.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:08 smithi111 ceph-mon[32784]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:40:08.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:08 smithi111 ceph-mon[32784]: from='osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100]' entity='osd.9' 2024-04-25T11:40:08.405 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:40:08 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:40:08.169+0000 7fd318f4f700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:40:08.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:08 smithi083 ceph-mon[29330]: from='osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T11:40:08.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:08 smithi083 ceph-mon[29330]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T11:40:08.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:08 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:40:08.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:08 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:40:08.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:08 smithi083 ceph-mon[29330]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:40:08.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:08 smithi083 ceph-mon[29330]: from='osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100]' entity='osd.9' 2024-04-25T11:40:08.418 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:08 smithi094 ceph-mon[33032]: from='osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-25T11:40:08.418 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:08 smithi094 ceph-mon[33032]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T11:40:08.418 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:08 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:40:08.418 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:08 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:40:08.418 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:08 smithi094 ceph-mon[33032]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:40:08.418 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:08 smithi094 ceph-mon[33032]: from='osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100]' entity='osd.9' 2024-04-25T11:40:08.745 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_1 2024-04-25T11:40:08.977 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:40:09.318 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:09 smithi094 ceph-mon[33032]: purged_snaps scrub starts 2024-04-25T11:40:09.319 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:09 smithi094 ceph-mon[33032]: purged_snaps scrub ok 2024-04-25T11:40:09.319 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:09 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:40:09.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:09 smithi111 ceph-mon[32784]: purged_snaps scrub starts 2024-04-25T11:40:09.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:09 smithi111 ceph-mon[32784]: purged_snaps scrub ok 2024-04-25T11:40:09.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:09 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:40:09.376 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:09 smithi083 ceph-mon[29330]: purged_snaps scrub starts 2024-04-25T11:40:09.377 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:09 smithi083 ceph-mon[29330]: purged_snaps scrub ok 2024-04-25T11:40:09.377 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:09 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:40:10.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100] boot 2024-04-25T11:40:10.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T11:40:10.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:40:10.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: Detected new or changed devices on smithi083 2024-04-25T11:40:10.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:10.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:10.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:10.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-25T11:40:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:40:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:40:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:10.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:10 smithi111 ceph-mon[32784]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:40:10.327 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100] boot 2024-04-25T11:40:10.327 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T11:40:10.328 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:40:10.328 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: Detected new or changed devices on smithi083 2024-04-25T11:40:10.328 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:10.328 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:10.328 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:10.328 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:10.328 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:10.328 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:10.328 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-25T11:40:10.328 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:10.328 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:10.328 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:40:10.328 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:10.328 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:10.329 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:10.329 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:40:10.329 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:10.329 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:10 smithi094 ceph-mon[33032]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:40:10.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: osd.9 [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100] boot 2024-04-25T11:40:10.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T11:40:10.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:40:10.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: Detected new or changed devices on smithi083 2024-04-25T11:40:10.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:10.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:10.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:10.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:10.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:10.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:10.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-25T11:40:10.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:10.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:10.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:40:10.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:10.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:10.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:10.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:40:10.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:10.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:10 smithi083 ceph-mon[29330]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T11:40:10.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.542+0000 7ff8de59c700 1 -- 172.21.15.94:0/4146475924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8d8108800 msgr2=0x7ff8d810ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:10.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.542+0000 7ff8de59c700 1 --2- 172.21.15.94:0/4146475924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8d8108800 0x7ff8d810ac10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000b0a0 tx=0x7ff8d000b3b0 comp rx=0 tx=0).stop 2024-04-25T11:40:10.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.542+0000 7ff8de59c700 1 -- 172.21.15.94:0/4146475924 shutdown_connections 2024-04-25T11:40:10.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.542+0000 7ff8de59c700 1 --2- 172.21.15.94:0/4146475924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8d8108800 0x7ff8d810ac10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:10.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.542+0000 7ff8de59c700 1 --2- 172.21.15.94:0/4146475924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d81035a0 0x7ff8d8105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:10.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.542+0000 7ff8de59c700 1 --2- 172.21.15.94:0/4146475924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8d8105ed0 0x7ff8d81082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:10.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.542+0000 7ff8de59c700 1 -- 172.21.15.94:0/4146475924 >> 172.21.15.94:0/4146475924 conn(0x7ff8d80fcf70 msgr2=0x7ff8d80ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:10.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.542+0000 7ff8de59c700 1 -- 172.21.15.94:0/4146475924 shutdown_connections 2024-04-25T11:40:10.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.542+0000 7ff8de59c700 1 -- 172.21.15.94:0/4146475924 wait complete. 2024-04-25T11:40:10.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.542+0000 7ff8de59c700 1 Processor -- start 2024-04-25T11:40:10.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.542+0000 7ff8de59c700 1 -- start start 2024-04-25T11:40:10.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8de59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8d81035a0 0x7ff8d8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:10.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8de59c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8d8105ed0 0x7ff8d8199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:10.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8de59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d8108800 0x7ff8d81a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:10.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8de59c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d810e540 con 0x7ff8d8108800 2024-04-25T11:40:10.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8de59c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d810e3c0 con 0x7ff8d81035a0 2024-04-25T11:40:10.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8de59c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d810e6c0 con 0x7ff8d8105ed0 2024-04-25T11:40:10.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8d7fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8d81035a0 0x7ff8d8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:10.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8d7fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8d81035a0 0x7ff8d8198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36452/0 (socket says 172.21.15.94:36452) 2024-04-25T11:40:10.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8d7fff700 1 -- 172.21.15.94:0/2660154253 learned_addr learned my addr 172.21.15.94:0/2660154253 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T11:40:10.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8dcb39700 1 --2- 172.21.15.94:0/2660154253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d8108800 0x7ff8d81a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:10.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8d77fe700 1 --2- 172.21.15.94:0/2660154253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8d8105ed0 0x7ff8d8199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:10.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8d7fff700 1 -- 172.21.15.94:0/2660154253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8d8105ed0 msgr2=0x7ff8d8199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:10.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8d7fff700 1 --2- 172.21.15.94:0/2660154253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8d8105ed0 0x7ff8d8199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:10.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8d7fff700 1 -- 172.21.15.94:0/2660154253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d8108800 msgr2=0x7ff8d81a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:10.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8d7fff700 1 --2- 172.21.15.94:0/2660154253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d8108800 0x7ff8d81a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:10.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8d7fff700 1 -- 172.21.15.94:0/2660154253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d81a0840 con 0x7ff8d81035a0 2024-04-25T11:40:10.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8d7fff700 1 --2- 172.21.15.94:0/2660154253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8d81035a0 0x7ff8d8198be0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800d950 tx=0x7ff8c800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:10.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8d57fa700 1 -- 172.21.15.94:0/2660154253 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8016070 con 0x7ff8d81035a0 2024-04-25T11:40:10.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8de59c700 1 -- 172.21.15.94:0/2660154253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d81a0b30 con 0x7ff8d81035a0 2024-04-25T11:40:10.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8d57fa700 1 -- 172.21.15.94:0/2660154253 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8c8009df0 con 0x7ff8d81035a0 2024-04-25T11:40:10.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8d57fa700 1 -- 172.21.15.94:0/2660154253 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c800f620 con 0x7ff8d81035a0 2024-04-25T11:40:10.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.543+0000 7ff8de59c700 1 -- 172.21.15.94:0/2660154253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d81a1070 con 0x7ff8d81035a0 2024-04-25T11:40:10.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.544+0000 7ff8de59c700 1 -- 172.21.15.94:0/2660154253 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8d8069730 con 0x7ff8d81035a0 2024-04-25T11:40:10.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.546+0000 7ff8d57fa700 1 -- 172.21.15.94:0/2660154253 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff8c8019070 con 0x7ff8d81035a0 2024-04-25T11:40:10.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.547+0000 7ff8d57fa700 1 --2- 172.21.15.94:0/2660154253 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff8c00a3c40 0x7ff8c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:10.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.547+0000 7ff8d57fa700 1 -- 172.21.15.94:0/2660154253 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7ff8c80cf700 con 0x7ff8d81035a0 2024-04-25T11:40:10.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.547+0000 7ff8d77fe700 1 --2- 172.21.15.94:0/2660154253 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff8c00a3c40 0x7ff8c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:10.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.547+0000 7ff8d77fe700 1 --2- 172.21.15.94:0/2660154253 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff8c00a3c40 0x7ff8c00a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc005950 tx=0x7ff8cc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:10.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.551+0000 7ff8d57fa700 1 -- 172.21.15.94:0/2660154253 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8c809c8e0 con 0x7ff8d81035a0 2024-04-25T11:40:10.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:10.757+0000 7ff8de59c700 1 -- 172.21.15.94:0/2660154253 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff8d8062e80 con 0x7ff8c00a3c40 2024-04-25T11:40:11.352 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:11 smithi094 ceph-mon[33032]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T11:40:11.352 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:11 smithi094 ceph-mon[33032]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:40:11.352 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:11 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:40:11.352 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:11 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:40:11.352 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:11 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:11.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:11 smithi083 ceph-mon[29330]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T11:40:11.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:11 smithi083 ceph-mon[29330]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:40:11.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:11 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:40:11.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:11 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:40:11.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:11 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:11.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:11 smithi111 ceph-mon[32784]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T11:40:11.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:11 smithi111 ceph-mon[32784]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:40:11.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:11 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:40:11.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:11 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:40:11.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:11 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:12.364 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:12 smithi094 ceph-mon[33032]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:12.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:12 smithi083 ceph-mon[29330]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:12.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:12 smithi111 ceph-mon[32784]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:13.767 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:13 smithi094 ceph-mon[33032]: from='client.? 172.21.15.94:0/2953299758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3d3f69-db95-47f3-aced-3eaa340839da"}]: dispatch 2024-04-25T11:40:13.767 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:13 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3d3f69-db95-47f3-aced-3eaa340839da"}]: dispatch 2024-04-25T11:40:13.767 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:13 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d3d3f69-db95-47f3-aced-3eaa340839da"}]': finished 2024-04-25T11:40:13.767 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:13 smithi094 ceph-mon[33032]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T11:40:13.767 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:13 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:13.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:13 smithi111 ceph-mon[32784]: from='client.? 172.21.15.94:0/2953299758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3d3f69-db95-47f3-aced-3eaa340839da"}]: dispatch 2024-04-25T11:40:13.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:13 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3d3f69-db95-47f3-aced-3eaa340839da"}]: dispatch 2024-04-25T11:40:13.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:13 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d3d3f69-db95-47f3-aced-3eaa340839da"}]': finished 2024-04-25T11:40:13.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:13 smithi111 ceph-mon[32784]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T11:40:13.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:13 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:13.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:13 smithi083 ceph-mon[29330]: from='client.? 172.21.15.94:0/2953299758' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3d3f69-db95-47f3-aced-3eaa340839da"}]: dispatch 2024-04-25T11:40:13.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:13 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d3d3f69-db95-47f3-aced-3eaa340839da"}]: dispatch 2024-04-25T11:40:13.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:13 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d3d3f69-db95-47f3-aced-3eaa340839da"}]': finished 2024-04-25T11:40:13.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:13 smithi083 ceph-mon[29330]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T11:40:13.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:13 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:14.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:14 smithi094 ceph-mon[33032]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:14.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:14 smithi094 ceph-mon[33032]: from='client.? 172.21.15.94:0/3466420878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:40:14.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:14 smithi111 ceph-mon[32784]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:14.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:14 smithi111 ceph-mon[32784]: from='client.? 172.21.15.94:0/3466420878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:40:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:14 smithi083 ceph-mon[29330]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:14.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:14 smithi083 ceph-mon[29330]: from='client.? 172.21.15.94:0/3466420878' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:40:17.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:16 smithi094 ceph-mon[33032]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:17.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:16 smithi111 ceph-mon[32784]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:17.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:16 smithi083 ceph-mon[29330]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:19.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:18 smithi094 ceph-mon[33032]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:19.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:18 smithi111 ceph-mon[32784]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:19.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:18 smithi083 ceph-mon[29330]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:21.208 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:20 smithi083 ceph-mon[29330]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:21.208 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T11:40:21.208 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:21.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:20 smithi094 ceph-mon[33032]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:21.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T11:40:21.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:21.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:20 smithi111 ceph-mon[32784]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:21.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T11:40:21.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:22.224 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:21 smithi083 ceph-mon[29330]: Deploying daemon osd.10 on smithi094 2024-04-25T11:40:22.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:21 smithi094 ceph-mon[33032]: Deploying daemon osd.10 on smithi094 2024-04-25T11:40:22.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:21 smithi111 ceph-mon[32784]: Deploying daemon osd.10 on smithi094 2024-04-25T11:40:23.140 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:22 smithi094 ceph-mon[33032]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:23.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:22 smithi111 ceph-mon[32784]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:23.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:22 smithi083 ceph-mon[29330]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:25.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:24 smithi094 ceph-mon[33032]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:25.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:24 smithi111 ceph-mon[32784]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:25.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:24 smithi083 ceph-mon[29330]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:27.038 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:26 smithi094 ceph-mon[33032]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:27.038 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:26 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:27.038 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:26 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:27.038 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:26 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:27.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:26 smithi111 ceph-mon[32784]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:27.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:26 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:27.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:26 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:27.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:26 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:26 smithi083 ceph-mon[29330]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:26 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:26 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:27.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:26 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:29.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:28 smithi094 ceph-mon[33032]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:29.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:28 smithi111 ceph-mon[32784]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:29.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:28 smithi083 ceph-mon[29330]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:29.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:29.869+0000 7ff8d57fa700 1 -- 172.21.15.94:0/2660154253 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff8d8062e80 con 0x7ff8c00a3c40 2024-04-25T11:40:29.856 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 10 on host 'smithi094' 2024-04-25T11:40:29.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:29.872+0000 7ff8de59c700 1 -- 172.21.15.94:0/2660154253 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff8c00a3c40 msgr2=0x7ff8c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:29.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:29.872+0000 7ff8de59c700 1 --2- 172.21.15.94:0/2660154253 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff8c00a3c40 0x7ff8c00a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc005950 tx=0x7ff8cc0058e0 comp rx=0 tx=0).stop 2024-04-25T11:40:29.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:29.872+0000 7ff8de59c700 1 -- 172.21.15.94:0/2660154253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8d81035a0 msgr2=0x7ff8d8198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:29.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:29.872+0000 7ff8de59c700 1 --2- 172.21.15.94:0/2660154253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8d81035a0 0x7ff8d8198be0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800d950 tx=0x7ff8c800dc60 comp rx=0 tx=0).stop 2024-04-25T11:40:29.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:29.872+0000 7ff8de59c700 1 -- 172.21.15.94:0/2660154253 shutdown_connections 2024-04-25T11:40:29.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:29.872+0000 7ff8de59c700 1 --2- 172.21.15.94:0/2660154253 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff8c00a3c40 0x7ff8c00a6100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:29.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:29.872+0000 7ff8de59c700 1 --2- 172.21.15.94:0/2660154253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8d8108800 0x7ff8d81a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:29.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:29.872+0000 7ff8de59c700 1 --2- 172.21.15.94:0/2660154253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8d81035a0 0x7ff8d8198be0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:29.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:29.872+0000 7ff8de59c700 1 --2- 172.21.15.94:0/2660154253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8d8105ed0 0x7ff8d8199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:29.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:29.872+0000 7ff8de59c700 1 -- 172.21.15.94:0/2660154253 >> 172.21.15.94:0/2660154253 conn(0x7ff8d80fcf70 msgr2=0x7ff8d8103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:29.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:29.872+0000 7ff8de59c700 1 -- 172.21.15.94:0/2660154253 shutdown_connections 2024-04-25T11:40:29.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T11:40:29.872+0000 7ff8de59c700 1 -- 172.21.15.94:0/2660154253 wait complete. 2024-04-25T11:40:30.540 DEBUG:teuthology.orchestra.run.smithi094:osd.10> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T11:40:30.542 INFO:tasks.cephadm:Deploying osd.11 on smithi111 with /dev/vg_nvme/lv_1... 2024-04-25T11:40:30.543 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T11:40:30.777 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:40:31.113 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:30 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:31.113 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:30 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:31.113 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:30 smithi111 ceph-mon[32784]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:31.113 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:30 smithi111 ceph-mon[32784]: from='osd.10 [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T11:40:31.113 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:30 smithi111 ceph-mon[32784]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T11:40:31.113 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:30 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:31.113 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:30 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:31.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:31.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:31.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:30 smithi083 ceph-mon[29330]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:31.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:30 smithi083 ceph-mon[29330]: from='osd.10 [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T11:40:31.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:30 smithi083 ceph-mon[29330]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T11:40:31.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:31.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:31.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:30 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:31.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:30 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:31.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:30 smithi094 ceph-mon[33032]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:31.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:30 smithi094 ceph-mon[33032]: from='osd.10 [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T11:40:31.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:30 smithi094 ceph-mon[33032]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T11:40:31.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:30 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:31.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:30 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:32.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:31 smithi083 ceph-mon[29330]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T11:40:32.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:31 smithi083 ceph-mon[29330]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T11:40:32.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:31 smithi083 ceph-mon[29330]: from='osd.10 [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:40:32.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:31 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:32.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:31 smithi083 ceph-mon[29330]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:40:32.171 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:31 smithi111 ceph-mon[32784]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T11:40:32.171 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:31 smithi111 ceph-mon[32784]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T11:40:32.171 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:31 smithi111 ceph-mon[32784]: from='osd.10 [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:40:32.171 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:31 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:32.171 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:31 smithi111 ceph-mon[32784]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:40:32.224 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:31 smithi094 ceph-mon[33032]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T11:40:32.224 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:31 smithi094 ceph-mon[33032]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T11:40:32.224 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:31 smithi094 ceph-mon[33032]: from='osd.10 [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:40:32.224 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:31 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:32.224 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:31 smithi094 ceph-mon[33032]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T11:40:32.729 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T11:40:32.730 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T11:40:32.730 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-25T11:40:32.730 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-25T11:40:32.730 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011153 s, 940 MB/s 2024-04-25T11:40:32.730 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-25T11:40:32.844 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:40:32 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:40:32.585+0000 7f3388fdd700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T11:40:32.844 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:40:32 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:40:32.600+0000 7f3380dc8700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:40:33.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:32 smithi083 ceph-mon[29330]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T11:40:33.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:32 smithi083 ceph-mon[29330]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T11:40:33.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:32 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:33.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:32 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:33.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:32 smithi083 ceph-mon[29330]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:33.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:32 smithi083 ceph-mon[29330]: from='osd.10 ' entity='osd.10' 2024-04-25T11:40:33.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:32 smithi094 ceph-mon[33032]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T11:40:33.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:32 smithi094 ceph-mon[33032]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T11:40:33.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:32 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:33.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:32 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:33.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:32 smithi094 ceph-mon[33032]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:33.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:32 smithi094 ceph-mon[33032]: from='osd.10 ' entity='osd.10' 2024-04-25T11:40:33.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:32 smithi111 ceph-mon[32784]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T11:40:33.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:32 smithi111 ceph-mon[32784]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T11:40:33.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:32 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:33.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:32 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:33.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:32 smithi111 ceph-mon[32784]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 894 GiB / 894 GiB avail 2024-04-25T11:40:33.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:32 smithi111 ceph-mon[32784]: from='osd.10 ' entity='osd.10' 2024-04-25T11:40:33.502 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_1 2024-04-25T11:40:33.738 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:40:33.966 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:33 smithi094 ceph-mon[33032]: purged_snaps scrub starts 2024-04-25T11:40:33.966 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:33 smithi094 ceph-mon[33032]: purged_snaps scrub ok 2024-04-25T11:40:33.967 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:33 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:33.967 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:33 smithi094 ceph-mon[33032]: osd.10 [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753] boot 2024-04-25T11:40:33.967 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:33 smithi094 ceph-mon[33032]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T11:40:33.967 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:33 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:34.126 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:33 smithi111 ceph-mon[32784]: purged_snaps scrub starts 2024-04-25T11:40:34.127 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:33 smithi111 ceph-mon[32784]: purged_snaps scrub ok 2024-04-25T11:40:34.127 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:33 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:34.127 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:33 smithi111 ceph-mon[32784]: osd.10 [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753] boot 2024-04-25T11:40:34.127 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:33 smithi111 ceph-mon[32784]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T11:40:34.127 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:33 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:34.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:33 smithi083 ceph-mon[29330]: purged_snaps scrub starts 2024-04-25T11:40:34.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:33 smithi083 ceph-mon[29330]: purged_snaps scrub ok 2024-04-25T11:40:34.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:33 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:34.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:33 smithi083 ceph-mon[29330]: osd.10 [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753] boot 2024-04-25T11:40:34.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:33 smithi083 ceph-mon[29330]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T11:40:34.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:33 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:40:35.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T11:40:35.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:35.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: Detected new or changed devices on smithi094 2024-04-25T11:40:35.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:35.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:35.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:35.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:35.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:35.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:35.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi094 to 4272M 2024-04-25T11:40:35.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:35.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:35.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:40:35.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:35.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:35.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:35.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:40:35.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:34 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:35.209 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T11:40:35.209 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:35.209 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: Detected new or changed devices on smithi094 2024-04-25T11:40:35.209 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:35.209 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:35.209 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:35.209 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:35.210 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:35.210 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:35.210 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi094 to 4272M 2024-04-25T11:40:35.210 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:35.210 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:35.210 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:40:35.210 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:35.210 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:35.210 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:35.210 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:40:35.210 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:34 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:35.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T11:40:35.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:35.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: Detected new or changed devices on smithi094 2024-04-25T11:40:35.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:35.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:35.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:35.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:35.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:35.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:40:35.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi094 to 4272M 2024-04-25T11:40:35.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:35.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:35.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:40:35.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:35.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:35.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:35.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:40:35.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:34 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:35.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.440+0000 7f4ffd6a7700 1 -- 172.21.15.111:0/585439331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ff8103ca0 msgr2=0x7f4ff8104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.440+0000 7f4ffd6a7700 1 --2- 172.21.15.111:0/585439331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ff8103ca0 0x7f4ff8104100 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009990 tx=0x7f4fec009ca0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.441+0000 7f4ffd6a7700 1 -- 172.21.15.111:0/585439331 shutdown_connections 2024-04-25T11:40:35.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.441+0000 7f4ffd6a7700 1 --2- 172.21.15.111:0/585439331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ff8104640 0x7f4ff810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.441+0000 7f4ffd6a7700 1 --2- 172.21.15.111:0/585439331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff8102aa0 0x7f4ff8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.441+0000 7f4ffd6a7700 1 --2- 172.21.15.111:0/585439331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ff8103ca0 0x7f4ff8104100 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.441+0000 7f4ffd6a7700 1 -- 172.21.15.111:0/585439331 >> 172.21.15.111:0/585439331 conn(0x7f4ff80fe040 msgr2=0x7f4ff8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:35.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.441+0000 7f4ffd6a7700 1 -- 172.21.15.111:0/585439331 shutdown_connections 2024-04-25T11:40:35.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.441+0000 7f4ffd6a7700 1 -- 172.21.15.111:0/585439331 wait complete. 2024-04-25T11:40:35.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.441+0000 7f4ffd6a7700 1 Processor -- start 2024-04-25T11:40:35.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.441+0000 7f4ffd6a7700 1 -- start start 2024-04-25T11:40:35.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.441+0000 7f4ffd6a7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff8102aa0 0x7f4ff80767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.441+0000 7f4ffd6a7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ff8103ca0 0x7f4ff8074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.441+0000 7f4ffd6a7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ff8104640 0x7f4ff8075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.441+0000 7f4ffd6a7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff810e5b0 con 0x7f4ff8104640 2024-04-25T11:40:35.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.441+0000 7f4ffd6a7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff810e430 con 0x7f4ff8102aa0 2024-04-25T11:40:35.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.441+0000 7f4ffd6a7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff810e730 con 0x7f4ff8103ca0 2024-04-25T11:40:35.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4ff67fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ff8103ca0 0x7f4ff8074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4ff67fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ff8103ca0 0x7f4ff8074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36480/0 (socket says 172.21.15.111:36480) 2024-04-25T11:40:35.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4ff67fc700 1 -- 172.21.15.111:0/546305636 learned_addr learned my addr 172.21.15.111:0/546305636 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-25T11:40:35.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4ff6ffd700 1 --2- 172.21.15.111:0/546305636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff8102aa0 0x7f4ff80767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4ff77fe700 1 --2- 172.21.15.111:0/546305636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ff8104640 0x7f4ff8075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4ff67fc700 1 -- 172.21.15.111:0/546305636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff8102aa0 msgr2=0x7f4ff80767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4ff67fc700 1 --2- 172.21.15.111:0/546305636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff8102aa0 0x7f4ff80767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4ff67fc700 1 -- 172.21.15.111:0/546305636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ff8104640 msgr2=0x7f4ff8075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:35.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4ff67fc700 1 --2- 172.21.15.111:0/546305636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ff8104640 0x7f4ff8075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:35.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4ff67fc700 1 -- 172.21.15.111:0/546305636 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff8075c20 con 0x7f4ff8103ca0 2024-04-25T11:40:35.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4ff67fc700 1 --2- 172.21.15.111:0/546305636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ff8103ca0 0x7f4ff8074e30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4fec006010 tx=0x7f4fec0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:35.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4fdffff700 1 -- 172.21.15.111:0/546305636 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec027070 con 0x7f4ff8103ca0 2024-04-25T11:40:35.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4ffd6a7700 1 -- 172.21.15.111:0/546305636 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff81a2d60 con 0x7f4ff8103ca0 2024-04-25T11:40:35.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4fdffff700 1 -- 172.21.15.111:0/546305636 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fec004000 con 0x7f4ff8103ca0 2024-04-25T11:40:35.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4fdffff700 1 -- 172.21.15.111:0/546305636 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec022cf0 con 0x7f4ff8103ca0 2024-04-25T11:40:35.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.442+0000 7f4ffd6a7700 1 -- 172.21.15.111:0/546305636 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff81a31b0 con 0x7f4ff8103ca0 2024-04-25T11:40:35.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.443+0000 7f4ffd6a7700 1 -- 172.21.15.111:0/546305636 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff802df80 con 0x7f4ff8103ca0 2024-04-25T11:40:35.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.445+0000 7f4fdffff700 1 -- 172.21.15.111:0/546305636 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4fec02a070 con 0x7f4ff8103ca0 2024-04-25T11:40:35.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.445+0000 7f4fdffff700 1 --2- 172.21.15.111:0/546305636 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4fe00a3c40 0x7f4fe00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:35.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.445+0000 7f4fdffff700 1 -- 172.21.15.111:0/546305636 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7f4fec0d14b0 con 0x7f4ff8103ca0 2024-04-25T11:40:35.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.445+0000 7f4ff6ffd700 1 --2- 172.21.15.111:0/546305636 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4fe00a3c40 0x7f4fe00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:35.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.446+0000 7f4ff6ffd700 1 --2- 172.21.15.111:0/546305636 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4fe00a3c40 0x7f4fe00a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8103b00 tx=0x7f4fe800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:35.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.449+0000 7f4fdffff700 1 -- 172.21.15.111:0/546305636 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4fec031070 con 0x7f4ff8103ca0 2024-04-25T11:40:35.642 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:35.654+0000 7f4ffd6a7700 1 -- 172.21.15.111:0/546305636 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4ff8062e80 con 0x7f4fe00a3c40 2024-04-25T11:40:36.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:35 smithi111 ceph-mon[32784]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T11:40:36.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:35 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:40:36.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:35 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:40:36.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:35 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:36.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:35 smithi083 ceph-mon[29330]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T11:40:36.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:35 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:40:36.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:35 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:40:36.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:35 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:36.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:35 smithi094 ceph-mon[33032]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T11:40:36.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:35 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T11:40:36.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:35 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T11:40:36.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:35 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:37.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:36 smithi083 ceph-mon[29330]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:40:37.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:36 smithi083 ceph-mon[29330]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T11:40:37.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:36 smithi083 ceph-mon[29330]: pgmap v239: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:37.237 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:36 smithi111 ceph-mon[32784]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:40:37.237 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:36 smithi111 ceph-mon[32784]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T11:40:37.237 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:36 smithi111 ceph-mon[32784]: pgmap v239: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:37.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:36 smithi094 ceph-mon[33032]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:40:37.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:36 smithi094 ceph-mon[33032]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T11:40:37.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:36 smithi094 ceph-mon[33032]: pgmap v239: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:38.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:37 smithi083 ceph-mon[29330]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T11:40:38.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:37 smithi094 ceph-mon[33032]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T11:40:38.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:37 smithi111 ceph-mon[32784]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T11:40:39.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:38 smithi111 ceph-mon[32784]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:39.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:38 smithi111 ceph-mon[32784]: from='client.? 172.21.15.111:0/1896366615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16ab0143-02e4-4a08-8cd3-3c9d9c841ad5"}]: dispatch 2024-04-25T11:40:39.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:38 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16ab0143-02e4-4a08-8cd3-3c9d9c841ad5"}]: dispatch 2024-04-25T11:40:39.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:38 smithi111 ceph-mon[32784]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16ab0143-02e4-4a08-8cd3-3c9d9c841ad5"}]': finished 2024-04-25T11:40:39.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:38 smithi111 ceph-mon[32784]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T11:40:39.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:38 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:40:39.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:38 smithi083 ceph-mon[29330]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:39.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:38 smithi083 ceph-mon[29330]: from='client.? 172.21.15.111:0/1896366615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16ab0143-02e4-4a08-8cd3-3c9d9c841ad5"}]: dispatch 2024-04-25T11:40:39.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:38 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16ab0143-02e4-4a08-8cd3-3c9d9c841ad5"}]: dispatch 2024-04-25T11:40:39.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:38 smithi083 ceph-mon[29330]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16ab0143-02e4-4a08-8cd3-3c9d9c841ad5"}]': finished 2024-04-25T11:40:39.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:38 smithi083 ceph-mon[29330]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T11:40:39.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:38 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:40:39.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:38 smithi094 ceph-mon[33032]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:39.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:38 smithi094 ceph-mon[33032]: from='client.? 172.21.15.111:0/1896366615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16ab0143-02e4-4a08-8cd3-3c9d9c841ad5"}]: dispatch 2024-04-25T11:40:39.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:38 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16ab0143-02e4-4a08-8cd3-3c9d9c841ad5"}]: dispatch 2024-04-25T11:40:39.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:38 smithi094 ceph-mon[33032]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16ab0143-02e4-4a08-8cd3-3c9d9c841ad5"}]': finished 2024-04-25T11:40:39.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:38 smithi094 ceph-mon[33032]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T11:40:39.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:38 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:40:40.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:39 smithi083 ceph-mon[29330]: from='client.? 172.21.15.111:0/2790839585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:40:40.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:39 smithi094 ceph-mon[33032]: from='client.? 172.21.15.111:0/2790839585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:40:40.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:39 smithi111 ceph-mon[32784]: from='client.? 172.21.15.111:0/2790839585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T11:40:41.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:40 smithi094 ceph-mon[33032]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T11:40:41.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:40 smithi111 ceph-mon[32784]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T11:40:41.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:40 smithi083 ceph-mon[29330]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T11:40:43.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:42 smithi094 ceph-mon[33032]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T11:40:43.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:42 smithi111 ceph-mon[32784]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T11:40:43.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:42 smithi083 ceph-mon[29330]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T11:40:45.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:44 smithi094 ceph-mon[33032]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T11:40:45.286 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:44 smithi111 ceph-mon[32784]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T11:40:45.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:44 smithi083 ceph-mon[29330]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T11:40:46.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:45 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T11:40:46.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:45 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:46.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:45 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T11:40:46.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:45 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:46.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:45 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T11:40:46.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:45 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:47.067 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:46 smithi111 ceph-mon[32784]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T11:40:47.068 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:46 smithi111 ceph-mon[32784]: Deploying daemon osd.11 on smithi111 2024-04-25T11:40:47.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:46 smithi094 ceph-mon[33032]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T11:40:47.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:46 smithi094 ceph-mon[33032]: Deploying daemon osd.11 on smithi111 2024-04-25T11:40:47.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:46 smithi083 ceph-mon[29330]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-25T11:40:47.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:46 smithi083 ceph-mon[29330]: Deploying daemon osd.11 on smithi111 2024-04-25T11:40:49.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:48 smithi094 ceph-mon[33032]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:40:49.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:48 smithi111 ceph-mon[32784]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:40:49.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:48 smithi083 ceph-mon[29330]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T11:40:51.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:50 smithi111 ceph-mon[32784]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-25T11:40:51.207 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:50 smithi083 ceph-mon[29330]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-25T11:40:51.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:50 smithi094 ceph-mon[33032]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-25T11:40:52.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:52.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:52.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:52 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:52.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:52 smithi083 ceph-mon[29330]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:52.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:52.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:52.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:52 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:52.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:52 smithi094 ceph-mon[33032]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:52.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:52.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:52.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:52 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:52.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:52 smithi111 ceph-mon[32784]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:55.235 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:54 smithi111 ceph-mon[32784]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:55.236 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:55.236 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:55.236 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:55.236 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:40:55.236 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:54 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:55.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:54 smithi094 ceph-mon[33032]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:55.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:55.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:55.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:55.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:40:55.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:54 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:55.307 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:55.321+0000 7f4fdffff700 1 -- 172.21.15.111:0/546305636 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4ff8062e80 con 0x7f4fe00a3c40 2024-04-25T11:40:55.308 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 11 on host 'smithi111' 2024-04-25T11:40:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:55.323+0000 7f4ffd6a7700 1 -- 172.21.15.111:0/546305636 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4fe00a3c40 msgr2=0x7f4fe00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:55.323+0000 7f4ffd6a7700 1 --2- 172.21.15.111:0/546305636 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4fe00a3c40 0x7f4fe00a6100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8103b00 tx=0x7f4fe800b410 comp rx=0 tx=0).stop 2024-04-25T11:40:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:55.323+0000 7f4ffd6a7700 1 -- 172.21.15.111:0/546305636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ff8103ca0 msgr2=0x7f4ff8074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:55.323+0000 7f4ffd6a7700 1 --2- 172.21.15.111:0/546305636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ff8103ca0 0x7f4ff8074e30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4fec006010 tx=0x7f4fec0047b0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:55.323+0000 7f4ffd6a7700 1 -- 172.21.15.111:0/546305636 shutdown_connections 2024-04-25T11:40:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:55.323+0000 7f4ffd6a7700 1 --2- 172.21.15.111:0/546305636 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4fe00a3c40 0x7f4fe00a6100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:55.323+0000 7f4ffd6a7700 1 --2- 172.21.15.111:0/546305636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ff8104640 0x7f4ff8075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:55.323+0000 7f4ffd6a7700 1 --2- 172.21.15.111:0/546305636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ff8102aa0 0x7f4ff80767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:55.323+0000 7f4ffd6a7700 1 --2- 172.21.15.111:0/546305636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ff8103ca0 0x7f4ff8074e30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:55.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:55.323+0000 7f4ffd6a7700 1 -- 172.21.15.111:0/546305636 >> 172.21.15.111:0/546305636 conn(0x7f4ff80fe040 msgr2=0x7f4ff81070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:55.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:55.323+0000 7f4ffd6a7700 1 -- 172.21.15.111:0/546305636 shutdown_connections 2024-04-25T11:40:55.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:40:55.323+0000 7f4ffd6a7700 1 -- 172.21.15.111:0/546305636 wait complete. 2024-04-25T11:40:55.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:54 smithi083 ceph-mon[29330]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:55.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:55.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:55.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:40:55.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:40:55.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:54 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:55.949 DEBUG:teuthology.orchestra.run.smithi111:osd.11> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T11:40:55.951 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T11:40:55.952 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T11:40:56.197 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:40:56.233 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:55 smithi111 ceph-mon[32784]: from='osd.11 [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T11:40:56.233 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:55 smithi111 ceph-mon[32784]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T11:40:56.233 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:55 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:56.233 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:55 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:56.233 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:55 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:56.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:55 smithi094 ceph-mon[33032]: from='osd.11 [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T11:40:56.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:55 smithi094 ceph-mon[33032]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T11:40:56.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:55 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:56.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:55 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:56.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:55 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:56.321 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:55 smithi083 ceph-mon[29330]: from='osd.11 [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T11:40:56.321 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:55 smithi083 ceph-mon[29330]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T11:40:56.321 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:55 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:56.322 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:55 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:40:56.322 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:55 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:40:57.250 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:56 smithi111 ceph-mon[32784]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:57.250 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:56 smithi111 ceph-mon[32784]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T11:40:57.250 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:56 smithi111 ceph-mon[32784]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T11:40:57.250 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:56 smithi111 ceph-mon[32784]: from='osd.11 [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:40:57.250 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:56 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:40:57.250 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:56 smithi111 ceph-mon[32784]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:40:57.251 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:40:56 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:40:56.963+0000 7fbe6ade7700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T11:40:57.251 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:40:56 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:40:56.978+0000 7fbe66bda700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:40:57.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:56 smithi094 ceph-mon[33032]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:57.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:56 smithi094 ceph-mon[33032]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T11:40:57.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:56 smithi094 ceph-mon[33032]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T11:40:57.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:56 smithi094 ceph-mon[33032]: from='osd.11 [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:40:57.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:56 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:40:57.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:56 smithi094 ceph-mon[33032]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:40:57.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:56 smithi083 ceph-mon[29330]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:57.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:56 smithi083 ceph-mon[29330]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T11:40:57.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:56 smithi083 ceph-mon[29330]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T11:40:57.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:56 smithi083 ceph-mon[29330]: from='osd.11 [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:40:57.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:56 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:40:57.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:56 smithi083 ceph-mon[29330]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-25T11:40:57.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.840+0000 7f2d1eb92700 1 -- 172.21.15.83:0/1695814373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d18076a10 msgr2=0x7f2d18074e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:57.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.840+0000 7f2d1eb92700 1 --2- 172.21.15.83:0/1695814373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d18076a10 0x7f2d18074e70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009990 tx=0x7f2d0c009ca0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.840+0000 7f2d1eb92700 1 -- 172.21.15.83:0/1695814373 shutdown_connections 2024-04-25T11:40:57.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.840+0000 7f2d1eb92700 1 --2- 172.21.15.83:0/1695814373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d1810a980 0x7f2d1810ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.840+0000 7f2d1eb92700 1 --2- 172.21.15.83:0/1695814373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d18076a10 0x7f2d18074e70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.840+0000 7f2d1eb92700 1 --2- 172.21.15.83:0/1695814373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d18075440 0x7f2d180758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.840+0000 7f2d1eb92700 1 -- 172.21.15.83:0/1695814373 >> 172.21.15.83:0/1695814373 conn(0x7f2d180fe040 msgr2=0x7f2d18100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:57.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.841+0000 7f2d1eb92700 1 -- 172.21.15.83:0/1695814373 shutdown_connections 2024-04-25T11:40:57.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.841+0000 7f2d1eb92700 1 -- 172.21.15.83:0/1695814373 wait complete. 2024-04-25T11:40:57.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.841+0000 7f2d1eb92700 1 Processor -- start 2024-04-25T11:40:57.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.841+0000 7f2d1eb92700 1 -- start start 2024-04-25T11:40:57.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.841+0000 7f2d1eb92700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d18075440 0x7f2d1819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.841+0000 7f2d1eb92700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d18076a10 0x7f2d1819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.841+0000 7f2d1eb92700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d1810a980 0x7f2d181a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.841+0000 7f2d1eb92700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18110770 con 0x7f2d18076a10 2024-04-25T11:40:57.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.841+0000 7f2d1eb92700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d181105f0 con 0x7f2d1810a980 2024-04-25T11:40:57.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.841+0000 7f2d1eb92700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2d181108f0 con 0x7f2d18075440 2024-04-25T11:40:57.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.841+0000 7f2d17fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d18076a10 0x7f2d1819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d17fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d18076a10 0x7f2d1819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54016/0 (socket says 172.21.15.83:54016) 2024-04-25T11:40:57.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d17fff700 1 -- 172.21.15.83:0/2967768109 learned_addr learned my addr 172.21.15.83:0/2967768109 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:40:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d1c92e700 1 --2- 172.21.15.83:0/2967768109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d18075440 0x7f2d1819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d17fff700 1 -- 172.21.15.83:0/2967768109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d18075440 msgr2=0x7f2d1819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d17fff700 1 --2- 172.21.15.83:0/2967768109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d18075440 0x7f2d1819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d17fff700 1 -- 172.21.15.83:0/2967768109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d1810a980 msgr2=0x7f2d181a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d1d12f700 1 --2- 172.21.15.83:0/2967768109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d1810a980 0x7f2d181a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d17fff700 1 --2- 172.21.15.83:0/2967768109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d1810a980 0x7f2d181a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d17fff700 1 -- 172.21.15.83:0/2967768109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d181a2ac0 con 0x7f2d18076a10 2024-04-25T11:40:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d1d12f700 1 --2- 172.21.15.83:0/2967768109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d1810a980 0x7f2d181a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:40:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d17fff700 1 --2- 172.21.15.83:0/2967768109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d18076a10 0x7f2d1819b3a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400b560 tx=0x7f2d0400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d15ffb700 1 -- 172.21.15.83:0/2967768109 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04015020 con 0x7f2d18076a10 2024-04-25T11:40:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d15ffb700 1 -- 172.21.15.83:0/2967768109 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d04011ce0 con 0x7f2d18076a10 2024-04-25T11:40:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d15ffb700 1 -- 172.21.15.83:0/2967768109 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04010da0 con 0x7f2d18076a10 2024-04-25T11:40:57.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d1eb92700 1 -- 172.21.15.83:0/2967768109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d181a2db0 con 0x7f2d18076a10 2024-04-25T11:40:57.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.842+0000 7f2d1eb92700 1 -- 172.21.15.83:0/2967768109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d181a32f0 con 0x7f2d18076a10 2024-04-25T11:40:57.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.843+0000 7f2d1eb92700 1 -- 172.21.15.83:0/2967768109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d18069730 con 0x7f2d18076a10 2024-04-25T11:40:57.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.844+0000 7f2d15ffb700 1 -- 172.21.15.83:0/2967768109 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2d04018020 con 0x7f2d18076a10 2024-04-25T11:40:57.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.845+0000 7f2d15ffb700 1 --2- 172.21.15.83:0/2967768109 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f2d080a3cc0 0x7f2d080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:40:57.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.845+0000 7f2d1c92e700 1 --2- 172.21.15.83:0/2967768109 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f2d080a3cc0 0x7f2d080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:40:57.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.845+0000 7f2d15ffb700 1 -- 172.21.15.83:0/2967768109 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f2d040cf5e0 con 0x7f2d18076a10 2024-04-25T11:40:57.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.845+0000 7f2d1c92e700 1 --2- 172.21.15.83:0/2967768109 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f2d080a3cc0 0x7f2d080a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00b5c0 tx=0x7f2d0c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:40:57.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:57.850+0000 7f2d15ffb700 1 -- 172.21.15.83:0/2967768109 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d0401f020 con 0x7f2d18076a10 2024-04-25T11:40:58.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:58.056+0000 7f2d1eb92700 1 -- 172.21.15.83:0/2967768109 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f2d1802e190 con 0x7f2d18076a10 2024-04-25T11:40:58.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:58.056+0000 7f2d15ffb700 1 -- 172.21.15.83:0/2967768109 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f2d0409c580 con 0x7f2d18076a10 2024-04-25T11:40:58.043 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:40:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:58.058+0000 7f2d1eb92700 1 -- 172.21.15.83:0/2967768109 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f2d080a3cc0 msgr2=0x7f2d080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:58.058+0000 7f2d1eb92700 1 --2- 172.21.15.83:0/2967768109 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f2d080a3cc0 0x7f2d080a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00b5c0 tx=0x7f2d0c005fd0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:58.058+0000 7f2d1eb92700 1 -- 172.21.15.83:0/2967768109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d18076a10 msgr2=0x7f2d1819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:40:58.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:58.058+0000 7f2d1eb92700 1 --2- 172.21.15.83:0/2967768109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d18076a10 0x7f2d1819b3a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400b560 tx=0x7f2d0400b920 comp rx=0 tx=0).stop 2024-04-25T11:40:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:58.058+0000 7f2d1eb92700 1 -- 172.21.15.83:0/2967768109 shutdown_connections 2024-04-25T11:40:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:58.058+0000 7f2d1eb92700 1 --2- 172.21.15.83:0/2967768109 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f2d080a3cc0 0x7f2d080a6180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:58.058+0000 7f2d1eb92700 1 --2- 172.21.15.83:0/2967768109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d1810a980 0x7f2d181a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:58.058+0000 7f2d1eb92700 1 --2- 172.21.15.83:0/2967768109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d18075440 0x7f2d1819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:58.058+0000 7f2d1eb92700 1 --2- 172.21.15.83:0/2967768109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d18076a10 0x7f2d1819b3a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:40:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:58.058+0000 7f2d1eb92700 1 -- 172.21.15.83:0/2967768109 >> 172.21.15.83:0/2967768109 conn(0x7f2d180fe040 msgr2=0x7f2d181068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:40:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:58.058+0000 7f2d1eb92700 1 -- 172.21.15.83:0/2967768109 shutdown_connections 2024-04-25T11:40:58.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:40:58.058+0000 7f2d1eb92700 1 -- 172.21.15.83:0/2967768109 wait complete. 2024-04-25T11:40:58.231 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:57 smithi111 ceph-mon[32784]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-25T11:40:58.231 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:57 smithi111 ceph-mon[32784]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T11:40:58.232 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:57 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:40:58.232 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:57 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:40:58.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:57 smithi094 ceph-mon[33032]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-25T11:40:58.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:57 smithi094 ceph-mon[33032]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T11:40:58.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:57 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:40:58.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:57 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:40:58.330 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:57 smithi083 ceph-mon[29330]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-25T11:40:58.331 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:57 smithi083 ceph-mon[29330]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T11:40:58.331 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:57 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:40:58.331 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:57 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:40:58.667 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714045257,"num_in_osds":12,"osd_in_since":1714045238,"num_remapped_pgs":0} 2024-04-25T11:40:58.668 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T11:40:58.906 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:40:59.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:58 smithi094 ceph-mon[33032]: purged_snaps scrub starts 2024-04-25T11:40:59.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:58 smithi094 ceph-mon[33032]: purged_snaps scrub ok 2024-04-25T11:40:59.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:58 smithi094 ceph-mon[33032]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:59.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:58 smithi094 ceph-mon[33032]: osd.11 [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468] boot 2024-04-25T11:40:59.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:58 smithi094 ceph-mon[33032]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T11:40:59.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:58 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:40:59.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:58 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2967768109' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T11:40:59.285 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:58 smithi083 ceph-mon[29330]: purged_snaps scrub starts 2024-04-25T11:40:59.285 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:58 smithi083 ceph-mon[29330]: purged_snaps scrub ok 2024-04-25T11:40:59.286 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:58 smithi083 ceph-mon[29330]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:59.286 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:58 smithi083 ceph-mon[29330]: osd.11 [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468] boot 2024-04-25T11:40:59.286 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:58 smithi083 ceph-mon[29330]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T11:40:59.286 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:58 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:40:59.286 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:58 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2967768109' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T11:40:59.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:58 smithi111 ceph-mon[32784]: purged_snaps scrub starts 2024-04-25T11:40:59.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:58 smithi111 ceph-mon[32784]: purged_snaps scrub ok 2024-04-25T11:40:59.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:58 smithi111 ceph-mon[32784]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T11:40:59.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:58 smithi111 ceph-mon[32784]: osd.11 [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468] boot 2024-04-25T11:40:59.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:58 smithi111 ceph-mon[32784]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T11:40:59.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:58 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:40:59.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:58 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2967768109' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T11:41:00.136 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:59 smithi083 ceph-mon[29330]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T11:41:00.136 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:59 smithi083 ceph-mon[29330]: Detected new or changed devices on smithi111 2024-04-25T11:41:00.136 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:59 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:00.136 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:59 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:00.136 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:59 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:41:00.137 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:59 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:41:00.137 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:59 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:41:00.137 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:59 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:41:00.137 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:59 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi111 to 4272M 2024-04-25T11:41:00.137 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:59 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:00.137 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:59 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:00.137 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:59 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:41:00.137 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:40:59 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:00.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:59 smithi094 ceph-mon[33032]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T11:41:00.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:59 smithi094 ceph-mon[33032]: Detected new or changed devices on smithi111 2024-04-25T11:41:00.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:59 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:00.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:59 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:00.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:59 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:41:00.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:59 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:41:00.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:59 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:41:00.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:59 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:41:00.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:59 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi111 to 4272M 2024-04-25T11:41:00.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:59 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:00.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:59 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:00.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:59 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:41:00.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:40:59 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:00.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:59 smithi111 ceph-mon[32784]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T11:41:00.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:59 smithi111 ceph-mon[32784]: Detected new or changed devices on smithi111 2024-04-25T11:41:00.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:59 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:00.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:59 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:00.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:59 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:41:00.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:59 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:41:00.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:59 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:41:00.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:59 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:41:00.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:59 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi111 to 4272M 2024-04-25T11:41:00.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:59 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:00.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:59 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:00.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:59 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:41:00.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:40:59 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:00.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.507+0000 7fb1d0784700 1 -- 172.21.15.83:0/1575967398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c806b1b0 msgr2=0x7fb1c806b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:00.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.507+0000 7fb1d0784700 1 --2- 172.21.15.83:0/1575967398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c806b1b0 0x7fb1c806b630 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80099c0 tx=0x7fb1b8009cd0 comp rx=0 tx=0).stop 2024-04-25T11:41:00.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.507+0000 7fb1d0784700 1 -- 172.21.15.83:0/1575967398 shutdown_connections 2024-04-25T11:41:00.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.507+0000 7fb1d0784700 1 --2- 172.21.15.83:0/1575967398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c806bb70 0x7fb1c810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:00.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.507+0000 7fb1d0784700 1 --2- 172.21.15.83:0/1575967398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1c8108a50 0x7fb1c8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:00.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.507+0000 7fb1d0784700 1 --2- 172.21.15.83:0/1575967398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c806b1b0 0x7fb1c806b630 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:00.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.507+0000 7fb1d0784700 1 -- 172.21.15.83:0/1575967398 >> 172.21.15.83:0/1575967398 conn(0x7fb1c8077990 msgr2=0x7fb1c8077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:00.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.508+0000 7fb1d0784700 1 -- 172.21.15.83:0/1575967398 shutdown_connections 2024-04-25T11:41:00.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.508+0000 7fb1d0784700 1 -- 172.21.15.83:0/1575967398 wait complete. 2024-04-25T11:41:00.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.508+0000 7fb1d0784700 1 Processor -- start 2024-04-25T11:41:00.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.508+0000 7fb1d0784700 1 -- start start 2024-04-25T11:41:00.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.508+0000 7fb1d0784700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c806b1b0 0x7fb1c8101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:00.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.508+0000 7fb1d0784700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1c806bb70 0x7fb1c8101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:00.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.508+0000 7fb1d0784700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c8108a50 0x7fb1c8102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:00.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.508+0000 7fb1d0784700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c8112c90 con 0x7fb1c806b1b0 2024-04-25T11:41:00.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.508+0000 7fb1d0784700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c8112b10 con 0x7fb1c8108a50 2024-04-25T11:41:00.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.508+0000 7fb1d0784700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c8112e10 con 0x7fb1c806bb70 2024-04-25T11:41:00.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.508+0000 7fb1ce520700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c806b1b0 0x7fb1c8101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:00.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.508+0000 7fb1ce520700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c806b1b0 0x7fb1c8101aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54038/0 (socket says 172.21.15.83:54038) 2024-04-25T11:41:00.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.508+0000 7fb1ce520700 1 -- 172.21.15.83:0/2197836817 learned_addr learned my addr 172.21.15.83:0/2197836817 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.508+0000 7fb1cdd1f700 1 --2- 172.21.15.83:0/2197836817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1c806bb70 0x7fb1c8101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.509+0000 7fb1ced21700 1 --2- 172.21.15.83:0/2197836817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c8108a50 0x7fb1c8102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.509+0000 7fb1ce520700 1 -- 172.21.15.83:0/2197836817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1c806bb70 msgr2=0x7fb1c8101fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.509+0000 7fb1ce520700 1 --2- 172.21.15.83:0/2197836817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1c806bb70 0x7fb1c8101fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.509+0000 7fb1ce520700 1 -- 172.21.15.83:0/2197836817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c8108a50 msgr2=0x7fb1c8102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.509+0000 7fb1ce520700 1 --2- 172.21.15.83:0/2197836817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c8108a50 0x7fb1c8102520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.509+0000 7fb1ce520700 1 -- 172.21.15.83:0/2197836817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c81a7770 con 0x7fb1c806b1b0 2024-04-25T11:41:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.509+0000 7fb1ce520700 1 --2- 172.21.15.83:0/2197836817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c806b1b0 0x7fb1c8101aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400d950 tx=0x7fb1c400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.509+0000 7fb1bf7fe700 1 -- 172.21.15.83:0/2197836817 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4016070 con 0x7fb1c806b1b0 2024-04-25T11:41:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.509+0000 7fb1d0784700 1 -- 172.21.15.83:0/2197836817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c81a7a60 con 0x7fb1c806b1b0 2024-04-25T11:41:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.509+0000 7fb1bf7fe700 1 -- 172.21.15.83:0/2197836817 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1c4009df0 con 0x7fb1c806b1b0 2024-04-25T11:41:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.509+0000 7fb1bf7fe700 1 -- 172.21.15.83:0/2197836817 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c400f3c0 con 0x7fb1c806b1b0 2024-04-25T11:41:00.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.509+0000 7fb1d0784700 1 -- 172.21.15.83:0/2197836817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c81a7fc0 con 0x7fb1c806b1b0 2024-04-25T11:41:00.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.510+0000 7fb1d0784700 1 -- 172.21.15.83:0/2197836817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c8069730 con 0x7fb1c806b1b0 2024-04-25T11:41:00.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.511+0000 7fb1bf7fe700 1 -- 172.21.15.83:0/2197836817 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb1c4019070 con 0x7fb1c806b1b0 2024-04-25T11:41:00.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.512+0000 7fb1bf7fe700 1 --2- 172.21.15.83:0/2197836817 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb1b40a3c40 0x7fb1b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:00.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.512+0000 7fb1cdd1f700 1 --2- 172.21.15.83:0/2197836817 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb1b40a3c40 0x7fb1b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:00.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.512+0000 7fb1bf7fe700 1 -- 172.21.15.83:0/2197836817 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb1c40cfbc0 con 0x7fb1c806b1b0 2024-04-25T11:41:00.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.512+0000 7fb1cdd1f700 1 --2- 172.21.15.83:0/2197836817 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb1b40a3c40 0x7fb1b40a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009990 tx=0x7fb1b800b790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:00.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.516+0000 7fb1bf7fe700 1 -- 172.21.15.83:0/2197836817 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb1c409c9a0 con 0x7fb1c806b1b0 2024-04-25T11:41:00.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.719+0000 7fb1d0784700 1 -- 172.21.15.83:0/2197836817 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1c802df80 con 0x7fb1c806b1b0 2024-04-25T11:41:00.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.720+0000 7fb1bf7fe700 1 -- 172.21.15.83:0/2197836817 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18384 (secure 0 0 0) 0x7fb1c409c2a0 con 0x7fb1c806b1b0 2024-04-25T11:41:00.707 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:00.707 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":82,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:40:58.967556+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:41:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.722+0000 7fb1d0784700 1 -- 172.21.15.83:0/2197836817 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb1b40a3c40 msgr2=0x7fb1b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.722+0000 7fb1d0784700 1 --2- 172.21.15.83:0/2197836817 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb1b40a3c40 0x7fb1b40a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009990 tx=0x7fb1b800b790 comp rx=0 tx=0).stop 2024-04-25T11:41:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.722+0000 7fb1d0784700 1 -- 172.21.15.83:0/2197836817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c806b1b0 msgr2=0x7fb1c8101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.722+0000 7fb1d0784700 1 --2- 172.21.15.83:0/2197836817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c806b1b0 0x7fb1c8101aa0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400d950 tx=0x7fb1c400dc60 comp rx=0 tx=0).stop 2024-04-25T11:41:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.722+0000 7fb1d0784700 1 -- 172.21.15.83:0/2197836817 shutdown_connections 2024-04-25T11:41:00.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.722+0000 7fb1d0784700 1 --2- 172.21.15.83:0/2197836817 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb1b40a3c40 0x7fb1b40a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.722+0000 7fb1d0784700 1 --2- 172.21.15.83:0/2197836817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb1c8108a50 0x7fb1c8102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.722+0000 7fb1d0784700 1 --2- 172.21.15.83:0/2197836817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1c806b1b0 0x7fb1c8101aa0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.722+0000 7fb1d0784700 1 --2- 172.21.15.83:0/2197836817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1c806bb70 0x7fb1c8101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.722+0000 7fb1d0784700 1 -- 172.21.15.83:0/2197836817 >> 172.21.15.83:0/2197836817 conn(0x7fb1c8077990 msgr2=0x7fb1c81007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.722+0000 7fb1d0784700 1 -- 172.21.15.83:0/2197836817 shutdown_connections 2024-04-25T11:41:00.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:00.722+0000 7fb1d0784700 1 -- 172.21.15.83:0/2197836817 wait complete. 2024-04-25T11:41:01.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:01 smithi094 ceph-mon[33032]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:01.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:01 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2197836817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:01.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:01 smithi111 ceph-mon[32784]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:01.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:01 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2197836817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:01.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:01 smithi083 ceph-mon[29330]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:01.373 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:01 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2197836817' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:01.401 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:37:15.914779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:41:01.401 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T11:41:01.637 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:41:03.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.200+0000 7fbba781f700 1 -- 172.21.15.83:0/3215759095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba0101400 msgr2=0x7fbba0101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:03.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.200+0000 7fbba781f700 1 --2- 172.21.15.83:0/3215759095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba0101400 0x7fbba0101880 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009990 tx=0x7fbb9c009ca0 comp rx=0 tx=0).stop 2024-04-25T11:41:03.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.200+0000 7fbba781f700 1 -- 172.21.15.83:0/3215759095 shutdown_connections 2024-04-25T11:41:03.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.200+0000 7fbba781f700 1 --2- 172.21.15.83:0/3215759095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbba0101dc0 0x7fbba010f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:03.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.200+0000 7fbba781f700 1 --2- 172.21.15.83:0/3215759095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbba0107b20 0x7fbba0107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:03.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.200+0000 7fbba781f700 1 --2- 172.21.15.83:0/3215759095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba0101400 0x7fbba0101880 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:03.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.200+0000 7fbba781f700 1 -- 172.21.15.83:0/3215759095 >> 172.21.15.83:0/3215759095 conn(0x7fbba0076c60 msgr2=0x7fbba0077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:03.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.201+0000 7fbba781f700 1 -- 172.21.15.83:0/3215759095 shutdown_connections 2024-04-25T11:41:03.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.201+0000 7fbba781f700 1 -- 172.21.15.83:0/3215759095 wait complete. 2024-04-25T11:41:03.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.201+0000 7fbba781f700 1 Processor -- start 2024-04-25T11:41:03.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.201+0000 7fbba781f700 1 -- start start 2024-04-25T11:41:03.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.201+0000 7fbba781f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbba0101400 0x7fbba0199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:03.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.201+0000 7fbba781f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbba0101dc0 0x7fbba01997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:03.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.201+0000 7fbba781f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba0107b20 0x7fbba019db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:03.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.201+0000 7fbba781f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbba0112d30 con 0x7fbba0107b20 2024-04-25T11:41:03.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.201+0000 7fbba781f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbba0112bb0 con 0x7fbba0101400 2024-04-25T11:41:03.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.201+0000 7fbba781f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbba0112eb0 con 0x7fbba0101dc0 2024-04-25T11:41:03.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.201+0000 7fbba5dbc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba0107b20 0x7fbba019db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:03.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.202+0000 7fbba5dbc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba0107b20 0x7fbba019db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53628/0 (socket says 172.21.15.83:53628) 2024-04-25T11:41:03.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.202+0000 7fbba5dbc700 1 -- 172.21.15.83:0/2269164749 learned_addr learned my addr 172.21.15.83:0/2269164749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:03.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.202+0000 7fbba4dba700 1 --2- 172.21.15.83:0/2269164749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbba0101dc0 0x7fbba01997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:03.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.202+0000 7fbba55bb700 1 --2- 172.21.15.83:0/2269164749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbba0101400 0x7fbba0199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:03.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.202+0000 7fbba5dbc700 1 -- 172.21.15.83:0/2269164749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbba0101dc0 msgr2=0x7fbba01997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:03.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.202+0000 7fbba5dbc700 1 --2- 172.21.15.83:0/2269164749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbba0101dc0 0x7fbba01997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:03.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.202+0000 7fbba5dbc700 1 -- 172.21.15.83:0/2269164749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbba0101400 msgr2=0x7fbba0199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:03.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.202+0000 7fbba5dbc700 1 --2- 172.21.15.83:0/2269164749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbba0101400 0x7fbba0199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:03.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.202+0000 7fbba5dbc700 1 -- 172.21.15.83:0/2269164749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba019e270 con 0x7fbba0107b20 2024-04-25T11:41:03.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.202+0000 7fbba5dbc700 1 --2- 172.21.15.83:0/2269164749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba0107b20 0x7fbba019db30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800b970 tx=0x7fbb9800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:03.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.202+0000 7fbb967fc700 1 -- 172.21.15.83:0/2269164749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9800d860 con 0x7fbba0107b20 2024-04-25T11:41:03.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.202+0000 7fbb967fc700 1 -- 172.21.15.83:0/2269164749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9800e430 con 0x7fbba0107b20 2024-04-25T11:41:03.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.202+0000 7fbba781f700 1 -- 172.21.15.83:0/2269164749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba019e560 con 0x7fbba0107b20 2024-04-25T11:41:03.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.202+0000 7fbb967fc700 1 -- 172.21.15.83:0/2269164749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98017660 con 0x7fbba0107b20 2024-04-25T11:41:03.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.202+0000 7fbba781f700 1 -- 172.21.15.83:0/2269164749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba01a5de0 con 0x7fbba0107b20 2024-04-25T11:41:03.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.203+0000 7fbba781f700 1 -- 172.21.15.83:0/2269164749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba0069730 con 0x7fbba0107b20 2024-04-25T11:41:03.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.204+0000 7fbb967fc700 1 -- 172.21.15.83:0/2269164749 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbb9800da00 con 0x7fbba0107b20 2024-04-25T11:41:03.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.205+0000 7fbb967fc700 1 --2- 172.21.15.83:0/2269164749 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbb8c0a3cc0 0x7fbb8c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:03.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.205+0000 7fbba55bb700 1 --2- 172.21.15.83:0/2269164749 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbb8c0a3cc0 0x7fbb8c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:03.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.205+0000 7fbb967fc700 1 -- 172.21.15.83:0/2269164749 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbb980cfa80 con 0x7fbba0107b20 2024-04-25T11:41:03.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.205+0000 7fbba55bb700 1 --2- 172.21.15.83:0/2269164749 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbb8c0a3cc0 0x7fbb8c0a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbba00781e0 tx=0x7fbb90009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:03.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.209+0000 7fbb967fc700 1 -- 172.21.15.83:0/2269164749 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb98013030 con 0x7fbba0107b20 2024-04-25T11:41:03.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:03 smithi094 ceph-mon[33032]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:03.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:03 smithi111 ceph-mon[32784]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:03.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.408+0000 7fbba781f700 1 -- 172.21.15.83:0/2269164749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbba002df80 con 0x7fbba0107b20 2024-04-25T11:41:03.395 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:03 smithi083 ceph-mon[29330]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:03.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.409+0000 7fbb967fc700 1 -- 172.21.15.83:0/2269164749 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fbb9809c900 con 0x7fbba0107b20 2024-04-25T11:41:03.396 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T11:41:03.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.411+0000 7fbba781f700 1 -- 172.21.15.83:0/2269164749 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbb8c0a3cc0 msgr2=0x7fbb8c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:03.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.411+0000 7fbba781f700 1 --2- 172.21.15.83:0/2269164749 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbb8c0a3cc0 0x7fbb8c0a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbba00781e0 tx=0x7fbb90009190 comp rx=0 tx=0).stop 2024-04-25T11:41:03.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.411+0000 7fbba781f700 1 -- 172.21.15.83:0/2269164749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba0107b20 msgr2=0x7fbba019db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:03.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.411+0000 7fbba781f700 1 --2- 172.21.15.83:0/2269164749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba0107b20 0x7fbba019db30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800b970 tx=0x7fbb9800bc80 comp rx=0 tx=0).stop 2024-04-25T11:41:03.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.411+0000 7fbba781f700 1 -- 172.21.15.83:0/2269164749 shutdown_connections 2024-04-25T11:41:03.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.411+0000 7fbba781f700 1 --2- 172.21.15.83:0/2269164749 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbb8c0a3cc0 0x7fbb8c0a6180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:03.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.411+0000 7fbba781f700 1 --2- 172.21.15.83:0/2269164749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba0107b20 0x7fbba019db30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:03.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.411+0000 7fbba781f700 1 --2- 172.21.15.83:0/2269164749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbba0101400 0x7fbba0199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:03.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.411+0000 7fbba781f700 1 --2- 172.21.15.83:0/2269164749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbba0101dc0 0x7fbba01997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:03.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.411+0000 7fbba781f700 1 -- 172.21.15.83:0/2269164749 >> 172.21.15.83:0/2269164749 conn(0x7fbba0076c60 msgr2=0x7fbba0100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:03.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.411+0000 7fbba781f700 1 -- 172.21.15.83:0/2269164749 shutdown_connections 2024-04-25T11:41:03.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:03.411+0000 7fbba781f700 1 -- 172.21.15.83:0/2269164749 wait complete. 2024-04-25T11:41:04.058 INFO:tasks.cephadm:Adding mds.a on smithi083 2024-04-25T11:41:04.059 INFO:tasks.cephadm:Adding mds.d on smithi083 2024-04-25T11:41:04.059 INFO:tasks.cephadm:Adding mds.g on smithi083 2024-04-25T11:41:04.059 INFO:tasks.cephadm:Adding mds.j on smithi083 2024-04-25T11:41:04.059 INFO:tasks.cephadm:Adding mds.b on smithi094 2024-04-25T11:41:04.059 INFO:tasks.cephadm:Adding mds.e on smithi094 2024-04-25T11:41:04.059 INFO:tasks.cephadm:Adding mds.h on smithi094 2024-04-25T11:41:04.059 INFO:tasks.cephadm:Adding mds.k on smithi094 2024-04-25T11:41:04.059 INFO:tasks.cephadm:Adding mds.c on smithi111 2024-04-25T11:41:04.059 INFO:tasks.cephadm:Adding mds.f on smithi111 2024-04-25T11:41:04.059 INFO:tasks.cephadm:Adding mds.i on smithi111 2024-04-25T11:41:04.059 INFO:tasks.cephadm:Adding mds.l on smithi111 2024-04-25T11:41:04.059 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l' 2024-04-25T11:41:04.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:04 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2269164749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:41:04.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:04 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2269164749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:41:04.309 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:41:04.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:04 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2269164749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:41:05.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:05 smithi111 ceph-mon[32784]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:05.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:05 smithi094 ceph-mon[33032]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:05.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:05 smithi083 ceph-mon[29330]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:05.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.945+0000 7f1bde3c7700 1 -- 172.21.15.111:0/801865665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bd8102bc0 msgr2=0x7f1bd8103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:05.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.945+0000 7f1bde3c7700 1 --2- 172.21.15.111:0/801865665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bd8102bc0 0x7f1bd8103020 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009990 tx=0x7f1bcc009ca0 comp rx=0 tx=0).stop 2024-04-25T11:41:05.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.945+0000 7f1bde3c7700 1 -- 172.21.15.111:0/801865665 shutdown_connections 2024-04-25T11:41:05.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.945+0000 7f1bde3c7700 1 --2- 172.21.15.111:0/801865665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd8103560 0x7f1bd8109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:05.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.945+0000 7f1bde3c7700 1 --2- 172.21.15.111:0/801865665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bd81019c0 0x7f1bd8101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:05.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.945+0000 7f1bde3c7700 1 --2- 172.21.15.111:0/801865665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bd8102bc0 0x7f1bd8103020 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:05.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.945+0000 7f1bde3c7700 1 -- 172.21.15.111:0/801865665 >> 172.21.15.111:0/801865665 conn(0x7f1bd80fcfa0 msgr2=0x7f1bd80ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:05.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.945+0000 7f1bde3c7700 1 -- 172.21.15.111:0/801865665 shutdown_connections 2024-04-25T11:41:05.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.945+0000 7f1bde3c7700 1 -- 172.21.15.111:0/801865665 wait complete. 2024-04-25T11:41:05.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.945+0000 7f1bde3c7700 1 Processor -- start 2024-04-25T11:41:05.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.945+0000 7f1bde3c7700 1 -- start start 2024-04-25T11:41:05.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bde3c7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bd81019c0 0x7f1bd806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:05.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bde3c7700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd8102bc0 0x7f1bd819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:05.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bde3c7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bd8103560 0x7f1bd81a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:05.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bde3c7700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd810d4d0 con 0x7f1bd8102bc0 2024-04-25T11:41:05.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bde3c7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd810d350 con 0x7f1bd81019c0 2024-04-25T11:41:05.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bde3c7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd810d650 con 0x7f1bd8103560 2024-04-25T11:41:05.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bdc964700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bd8103560 0x7f1bd81a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:05.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bdc964700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bd8103560 0x7f1bd81a2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56410/0 (socket says 172.21.15.111:56410) 2024-04-25T11:41:05.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bdc964700 1 -- 172.21.15.111:0/1343954649 learned_addr learned my addr 172.21.15.111:0/1343954649 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-25T11:41:05.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bd77fe700 1 --2- 172.21.15.111:0/1343954649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd8102bc0 0x7f1bd819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:05.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bd7fff700 1 --2- 172.21.15.111:0/1343954649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bd81019c0 0x7f1bd806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:05.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bdc964700 1 -- 172.21.15.111:0/1343954649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bd81019c0 msgr2=0x7f1bd806bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:05.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bdc964700 1 --2- 172.21.15.111:0/1343954649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bd81019c0 0x7f1bd806bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:05.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bdc964700 1 -- 172.21.15.111:0/1343954649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd8102bc0 msgr2=0x7f1bd819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:05.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bdc964700 1 --2- 172.21.15.111:0/1343954649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd8102bc0 0x7f1bd819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:05.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bdc964700 1 -- 172.21.15.111:0/1343954649 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd81a2a50 con 0x7f1bd8103560 2024-04-25T11:41:05.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bd77fe700 1 --2- 172.21.15.111:0/1343954649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd8102bc0 0x7f1bd819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:41:05.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bdc964700 1 --2- 172.21.15.111:0/1343954649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bd8103560 0x7f1bd81a2270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0007d70 tx=0x7f1bd000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:05.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bd57fa700 1 -- 172.21.15.111:0/1343954649 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0016020 con 0x7f1bd8103560 2024-04-25T11:41:05.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bd57fa700 1 -- 172.21.15.111:0/1343954649 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd000ac90 con 0x7f1bd8103560 2024-04-25T11:41:05.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bde3c7700 1 -- 172.21.15.111:0/1343954649 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd81a2d40 con 0x7f1bd8103560 2024-04-25T11:41:05.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.946+0000 7f1bd57fa700 1 -- 172.21.15.111:0/1343954649 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd00115e0 con 0x7f1bd8103560 2024-04-25T11:41:05.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.947+0000 7f1bde3c7700 1 -- 172.21.15.111:0/1343954649 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd81a3230 con 0x7f1bd8103560 2024-04-25T11:41:05.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.948+0000 7f1bbeffd700 1 -- 172.21.15.111:0/1343954649 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd8069730 con 0x7f1bd8103560 2024-04-25T11:41:05.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.948+0000 7f1bd57fa700 1 -- 172.21.15.111:0/1343954649 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1bd0019020 con 0x7f1bd8103560 2024-04-25T11:41:05.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.949+0000 7f1bd57fa700 1 --2- 172.21.15.111:0/1343954649 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1bc00a3c40 0x7f1bc00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:05.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.949+0000 7f1bd57fa700 1 -- 172.21.15.111:0/1343954649 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1bd00cfec0 con 0x7f1bd8103560 2024-04-25T11:41:05.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.950+0000 7f1bd7fff700 1 --2- 172.21.15.111:0/1343954649 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1bc00a3c40 0x7f1bc00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:05.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.950+0000 7f1bd7fff700 1 --2- 172.21.15.111:0/1343954649 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1bc00a3c40 0x7f1bc00a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8102a20 tx=0x7f1bc8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:05.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:05.954+0000 7f1bd57fa700 1 -- 172.21.15.111:0/1343954649 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1bd0020020 con 0x7f1bd8103560 2024-04-25T11:41:06.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:06.168+0000 7f1bbeffd700 1 -- 172.21.15.111:0/1343954649 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l", "target": ["mon-mgr", ""]}) -- 0x7f1bd8063080 con 0x7f1bc00a3c40 2024-04-25T11:41:06.165 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:06.178+0000 7f1bd57fa700 1 -- 172.21.15.111:0/1343954649 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f1bd8063080 con 0x7f1bc00a3c40 2024-04-25T11:41:06.165 INFO:teuthology.orchestra.run.smithi111.stdout:Scheduled mds.all update... 2024-04-25T11:41:06.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:06.180+0000 7f1bbeffd700 1 -- 172.21.15.111:0/1343954649 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1bc00a3c40 msgr2=0x7f1bc00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:06.180+0000 7f1bbeffd700 1 --2- 172.21.15.111:0/1343954649 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1bc00a3c40 0x7f1bc00a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8102a20 tx=0x7f1bc8009190 comp rx=0 tx=0).stop 2024-04-25T11:41:06.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:06.180+0000 7f1bbeffd700 1 -- 172.21.15.111:0/1343954649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bd8103560 msgr2=0x7f1bd81a2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:06.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:06.180+0000 7f1bbeffd700 1 --2- 172.21.15.111:0/1343954649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bd8103560 0x7f1bd81a2270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0007d70 tx=0x7f1bd000a340 comp rx=0 tx=0).stop 2024-04-25T11:41:06.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:06.180+0000 7f1bbeffd700 1 -- 172.21.15.111:0/1343954649 shutdown_connections 2024-04-25T11:41:06.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:06.180+0000 7f1bbeffd700 1 --2- 172.21.15.111:0/1343954649 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1bc00a3c40 0x7f1bc00a6100 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.167 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:06.180+0000 7f1bbeffd700 1 --2- 172.21.15.111:0/1343954649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bd8103560 0x7f1bd81a2270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:06.180+0000 7f1bbeffd700 1 --2- 172.21.15.111:0/1343954649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bd81019c0 0x7f1bd806bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:06.180+0000 7f1bbeffd700 1 --2- 172.21.15.111:0/1343954649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1bd8102bc0 0x7f1bd819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:06.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:06.180+0000 7f1bbeffd700 1 -- 172.21.15.111:0/1343954649 >> 172.21.15.111:0/1343954649 conn(0x7f1bd80fcfa0 msgr2=0x7f1bd8105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:06.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:06.180+0000 7f1bbeffd700 1 -- 172.21.15.111:0/1343954649 shutdown_connections 2024-04-25T11:41:06.168 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-25T11:41:06.180+0000 7f1bbeffd700 1 -- 172.21.15.111:0/1343954649 wait complete. 2024-04-25T11:41:06.792 DEBUG:teuthology.orchestra.run.smithi083:mds.a> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T11:41:06.795 DEBUG:teuthology.orchestra.run.smithi083:mds.d> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T11:41:06.837 DEBUG:teuthology.orchestra.run.smithi083:mds.g> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T11:41:06.880 DEBUG:teuthology.orchestra.run.smithi083:mds.j> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T11:41:06.923 DEBUG:teuthology.orchestra.run.smithi094:mds.b> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T11:41:06.926 DEBUG:teuthology.orchestra.run.smithi094:mds.e> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T11:41:06.969 DEBUG:teuthology.orchestra.run.smithi094:mds.h> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T11:41:07.012 DEBUG:teuthology.orchestra.run.smithi094:mds.k> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T11:41:07.055 DEBUG:teuthology.orchestra.run.smithi111:mds.c> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T11:41:07.057 DEBUG:teuthology.orchestra.run.smithi111:mds.f> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T11:41:07.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:07 smithi111 ceph-mon[32784]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:07.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:07 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:07.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:07 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:41:07.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:07 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:07.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:07 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:41:07.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:07 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:07.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:07 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:07.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:07 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:07.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:07 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:07.072 DEBUG:teuthology.orchestra.run.smithi111:mds.i> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T11:41:07.115 DEBUG:teuthology.orchestra.run.smithi111:mds.l> sudo journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T11:41:07.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:07 smithi083 ceph-mon[29330]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:07.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:07 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:07.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:07 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:41:07.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:07 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:07.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:07 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:41:07.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:07 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:07.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:07 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:07.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:07 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:07.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:07 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:07.158 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T11:41:07.159 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:41:07.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 -- 172.21.15.83:0/278893763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3e4107cf0 msgr2=0x7ff3e410a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:07.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 --2- 172.21.15.83:0/278893763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3e4107cf0 0x7ff3e410a140 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009ca0 tx=0x7ff3dc02a8c0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 -- 172.21.15.83:0/278893763 shutdown_connections 2024-04-25T11:41:07.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 --2- 172.21.15.83:0/278893763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3e4107cf0 0x7ff3e410a140 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 --2- 172.21.15.83:0/278893763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e41053c0 0x7ff3e41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 --2- 172.21.15.83:0/278893763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e4102a90 0x7ff3e4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 -- 172.21.15.83:0/278893763 >> 172.21.15.83:0/278893763 conn(0x7ff3e40fc4a0 msgr2=0x7ff3e40fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 -- 172.21.15.83:0/278893763 shutdown_connections 2024-04-25T11:41:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 -- 172.21.15.83:0/278893763 wait complete. 2024-04-25T11:41:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 Processor -- start 2024-04-25T11:41:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 -- start start 2024-04-25T11:41:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e4102a90 0x7ff3e4102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3e41053c0 0x7ff3e4100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e4107cf0 0x7ff3e4100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e410db30 con 0x7ff3e41053c0 2024-04-25T11:41:07.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e410d9b0 con 0x7ff3e4107cf0 2024-04-25T11:41:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3eabad640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e410dcb0 con 0x7ff3e4102a90 2024-04-25T11:41:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3e41053c0 0x7ff3e4100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3e41053c0 0x7ff3e4100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53642/0 (socket says 172.21.15.83:53642) 2024-04-25T11:41:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3e3fff640 1 -- 172.21.15.83:0/153170843 learned_addr learned my addr 172.21.15.83:0/153170843 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.276+0000 7ff3e8949640 1 --2- 172.21.15.83:0/153170843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e4102a90 0x7ff3e4102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.277+0000 7ff3e3fff640 1 -- 172.21.15.83:0/153170843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e4102a90 msgr2=0x7ff3e4102400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.277+0000 7ff3e3fff640 1 --2- 172.21.15.83:0/153170843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e4102a90 0x7ff3e4102400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.277+0000 7ff3e3fff640 1 -- 172.21.15.83:0/153170843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e4107cf0 msgr2=0x7ff3e4100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.277+0000 7ff3e3fff640 1 --2- 172.21.15.83:0/153170843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e4107cf0 0x7ff3e4100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.277+0000 7ff3e3fff640 1 -- 172.21.15.83:0/153170843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e4101810 con 0x7ff3e41053c0 2024-04-25T11:41:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.277+0000 7ff3e3fff640 1 --2- 172.21.15.83:0/153170843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3e41053c0 0x7ff3e4100a40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c970 tx=0x7ff3d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.277+0000 7ff3e1ffb640 1 -- 172.21.15.83:0/153170843 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8007bb0 con 0x7ff3e41053c0 2024-04-25T11:41:07.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.277+0000 7ff3e1ffb640 1 -- 172.21.15.83:0/153170843 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3d8007d50 con 0x7ff3e41053c0 2024-04-25T11:41:07.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.277+0000 7ff3eabad640 1 -- 172.21.15.83:0/153170843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e41a22d0 con 0x7ff3e41053c0 2024-04-25T11:41:07.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.277+0000 7ff3e1ffb640 1 -- 172.21.15.83:0/153170843 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d800f450 con 0x7ff3e41053c0 2024-04-25T11:41:07.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.277+0000 7ff3eabad640 1 -- 172.21.15.83:0/153170843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e41a2820 con 0x7ff3e41053c0 2024-04-25T11:41:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.278+0000 7ff3eabad640 1 -- 172.21.15.83:0/153170843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e40fdf90 con 0x7ff3e41053c0 2024-04-25T11:41:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.279+0000 7ff3e1ffb640 1 -- 172.21.15.83:0/153170843 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff3d8004f20 con 0x7ff3e41053c0 2024-04-25T11:41:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.280+0000 7ff3e1ffb640 1 --2- 172.21.15.83:0/153170843 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff3cc0a3bb0 0x7ff3cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.280+0000 7ff3e8949640 1 --2- 172.21.15.83:0/153170843 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff3cc0a3bb0 0x7ff3cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.280+0000 7ff3e1ffb640 1 -- 172.21.15.83:0/153170843 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff3d80d0000 con 0x7ff3e41053c0 2024-04-25T11:41:07.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.280+0000 7ff3e8949640 1 --2- 172.21.15.83:0/153170843 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff3cc0a3bb0 0x7ff3cc0a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009bd0 tx=0x7ff3d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:07.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.285+0000 7ff3e1ffb640 1 -- 172.21.15.83:0/153170843 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff3d8014030 con 0x7ff3e41053c0 2024-04-25T11:41:07.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:07 smithi094 ceph-mon[33032]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:07.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:07 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:07.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:07 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:41:07.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:07 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:07.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:07 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:41:07.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:07 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:07.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:07 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:07.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:07 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:07.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:07 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:07.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.464+0000 7ff3eabad640 1 -- 172.21.15.83:0/153170843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3e402d770 con 0x7ff3e41053c0 2024-04-25T11:41:07.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.465+0000 7ff3e1ffb640 1 -- 172.21.15.83:0/153170843 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18384 (secure 0 0 0) 0x7ff3d809ce80 con 0x7ff3e41053c0 2024-04-25T11:41:07.453 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:07.453 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":82,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:40:58.967556+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:41:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.467+0000 7ff3eabad640 1 -- 172.21.15.83:0/153170843 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff3cc0a3bb0 msgr2=0x7ff3cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.467+0000 7ff3eabad640 1 --2- 172.21.15.83:0/153170843 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff3cc0a3bb0 0x7ff3cc0a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009bd0 tx=0x7ff3d4005f90 comp rx=0 tx=0).stop 2024-04-25T11:41:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.467+0000 7ff3eabad640 1 -- 172.21.15.83:0/153170843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3e41053c0 msgr2=0x7ff3e4100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.467+0000 7ff3eabad640 1 --2- 172.21.15.83:0/153170843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3e41053c0 0x7ff3e4100a40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800c970 tx=0x7ff3d800ce30 comp rx=0 tx=0).stop 2024-04-25T11:41:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.467+0000 7ff3eabad640 1 -- 172.21.15.83:0/153170843 shutdown_connections 2024-04-25T11:41:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.467+0000 7ff3eabad640 1 --2- 172.21.15.83:0/153170843 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff3cc0a3bb0 0x7ff3cc0a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.467+0000 7ff3eabad640 1 --2- 172.21.15.83:0/153170843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3e41053c0 0x7ff3e4100a40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.467+0000 7ff3eabad640 1 --2- 172.21.15.83:0/153170843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3e4107cf0 0x7ff3e4100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.467+0000 7ff3eabad640 1 --2- 172.21.15.83:0/153170843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3e4102a90 0x7ff3e4102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.467+0000 7ff3eabad640 1 -- 172.21.15.83:0/153170843 >> 172.21.15.83:0/153170843 conn(0x7ff3e40fc4a0 msgr2=0x7ff3e41034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:07.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.467+0000 7ff3eabad640 1 -- 172.21.15.83:0/153170843 shutdown_connections 2024-04-25T11:41:07.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.467+0000 7ff3eabad640 1 -- 172.21.15.83:0/153170843 wait complete. 2024-04-25T11:41:07.466 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:37:15.914779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:41:07.467 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:41:07.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.579+0000 7f8f6f394640 1 -- 172.21.15.83:0/929367765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f68101fe0 msgr2=0x7f8f68102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.579+0000 7f8f6f394640 1 --2- 172.21.15.83:0/929367765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f68101fe0 0x7f8f68102460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009900 tx=0x7f8f6402a410 comp rx=0 tx=0).stop 2024-04-25T11:41:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.579+0000 7f8f6f394640 1 -- 172.21.15.83:0/929367765 shutdown_connections 2024-04-25T11:41:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.579+0000 7f8f6f394640 1 --2- 172.21.15.83:0/929367765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f68101fe0 0x7f8f68102460 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.579+0000 7f8f6f394640 1 --2- 172.21.15.83:0/929367765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f681029a0 0x7f8f681092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.579+0000 7f8f6f394640 1 --2- 172.21.15.83:0/929367765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f68100e90 0x7f8f681012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.579+0000 7f8f6f394640 1 -- 172.21.15.83:0/929367765 >> 172.21.15.83:0/929367765 conn(0x7f8f680fc400 msgr2=0x7f8f680fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.580+0000 7f8f6f394640 1 -- 172.21.15.83:0/929367765 shutdown_connections 2024-04-25T11:41:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.580+0000 7f8f6f394640 1 -- 172.21.15.83:0/929367765 wait complete. 2024-04-25T11:41:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.580+0000 7f8f6f394640 1 Processor -- start 2024-04-25T11:41:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.580+0000 7f8f6f394640 1 -- start start 2024-04-25T11:41:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.580+0000 7f8f6f394640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f68100e90 0x7f8f681982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:07.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.580+0000 7f8f6f394640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f68101fe0 0x7f8f68198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.580+0000 7f8f6f394640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f681029a0 0x7f8f6819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.580+0000 7f8f6f394640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6810ca40 con 0x7f8f68101fe0 2024-04-25T11:41:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.580+0000 7f8f6f394640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6810c8c0 con 0x7f8f68100e90 2024-04-25T11:41:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.580+0000 7f8f6f394640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6810cbc0 con 0x7f8f681029a0 2024-04-25T11:41:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.580+0000 7f8f6c92f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f68101fe0 0x7f8f68198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.580+0000 7f8f6c92f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f68101fe0 0x7f8f68198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53664/0 (socket says 172.21.15.83:53664) 2024-04-25T11:41:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.580+0000 7f8f6c92f640 1 -- 172.21.15.83:0/4245298134 learned_addr learned my addr 172.21.15.83:0/4245298134 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.580+0000 7f8f6d130640 1 --2- 172.21.15.83:0/4245298134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f68100e90 0x7f8f681982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.580+0000 7f8f6d931640 1 --2- 172.21.15.83:0/4245298134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f681029a0 0x7f8f6819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.581+0000 7f8f6c92f640 1 -- 172.21.15.83:0/4245298134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f681029a0 msgr2=0x7f8f6819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.581+0000 7f8f6c92f640 1 --2- 172.21.15.83:0/4245298134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f681029a0 0x7f8f6819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.581+0000 7f8f6c92f640 1 -- 172.21.15.83:0/4245298134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f68100e90 msgr2=0x7f8f681982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.581+0000 7f8f6c92f640 1 --2- 172.21.15.83:0/4245298134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f68100e90 0x7f8f681982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.581+0000 7f8f6c92f640 1 -- 172.21.15.83:0/4245298134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6819ff10 con 0x7f8f68101fe0 2024-04-25T11:41:07.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.581+0000 7f8f6d130640 1 --2- 172.21.15.83:0/4245298134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f68100e90 0x7f8f681982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:07.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.581+0000 7f8f6c92f640 1 --2- 172.21.15.83:0/4245298134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f68101fe0 0x7f8f68198830 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8f64006010 tx=0x7f8f640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:07.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.581+0000 7f8f5e7fc640 1 -- 172.21.15.83:0/4245298134 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6403f030 con 0x7f8f68101fe0 2024-04-25T11:41:07.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.581+0000 7f8f5e7fc640 1 -- 172.21.15.83:0/4245298134 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f640318f0 con 0x7f8f68101fe0 2024-04-25T11:41:07.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.581+0000 7f8f6f394640 1 -- 172.21.15.83:0/4245298134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f681a01a0 con 0x7f8f68101fe0 2024-04-25T11:41:07.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.581+0000 7f8f5e7fc640 1 -- 172.21.15.83:0/4245298134 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64035070 con 0x7f8f68101fe0 2024-04-25T11:41:07.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.581+0000 7f8f6f394640 1 -- 172.21.15.83:0/4245298134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f681a0710 con 0x7f8f68101fe0 2024-04-25T11:41:07.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.582+0000 7f8f6f394640 1 -- 172.21.15.83:0/4245298134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f681a0970 con 0x7f8f68101fe0 2024-04-25T11:41:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.583+0000 7f8f5e7fc640 1 -- 172.21.15.83:0/4245298134 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8f64042030 con 0x7f8f68101fe0 2024-04-25T11:41:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.583+0000 7f8f5e7fc640 1 --2- 172.21.15.83:0/4245298134 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8f540a3bb0 0x7f8f540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.584+0000 7f8f6d130640 1 --2- 172.21.15.83:0/4245298134 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8f540a3bb0 0x7f8f540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.584+0000 7f8f5e7fc640 1 -- 172.21.15.83:0/4245298134 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8f640f0570 con 0x7f8f68101fe0 2024-04-25T11:41:07.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.584+0000 7f8f6d130640 1 --2- 172.21.15.83:0/4245298134 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8f540a3bb0 0x7f8f540a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009930 tx=0x7f8f58008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.588+0000 7f8f5e7fc640 1 -- 172.21.15.83:0/4245298134 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f640bd3f0 con 0x7f8f68101fe0 2024-04-25T11:41:07.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.763+0000 7f8f6f394640 1 -- 172.21.15.83:0/4245298134 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8f681a0d00 con 0x7f8f68101fe0 2024-04-25T11:41:07.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.763+0000 7f8f5e7fc640 1 -- 172.21.15.83:0/4245298134 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f8f640bccf0 con 0x7f8f68101fe0 2024-04-25T11:41:07.754 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T11:41:07.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.765+0000 7f8f6f394640 1 -- 172.21.15.83:0/4245298134 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8f540a3bb0 msgr2=0x7f8f540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:07.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.765+0000 7f8f6f394640 1 --2- 172.21.15.83:0/4245298134 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8f540a3bb0 0x7f8f540a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009930 tx=0x7f8f58008040 comp rx=0 tx=0).stop 2024-04-25T11:41:07.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.765+0000 7f8f6f394640 1 -- 172.21.15.83:0/4245298134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f68101fe0 msgr2=0x7f8f68198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:07.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.765+0000 7f8f6f394640 1 --2- 172.21.15.83:0/4245298134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f68101fe0 0x7f8f68198830 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8f64006010 tx=0x7f8f640042f0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.765+0000 7f8f6f394640 1 -- 172.21.15.83:0/4245298134 shutdown_connections 2024-04-25T11:41:07.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.765+0000 7f8f6f394640 1 --2- 172.21.15.83:0/4245298134 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8f540a3bb0 0x7f8f540a6070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.765+0000 7f8f6f394640 1 --2- 172.21.15.83:0/4245298134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f68101fe0 0x7f8f68198830 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.765+0000 7f8f6f394640 1 --2- 172.21.15.83:0/4245298134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f681029a0 0x7f8f6819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.765+0000 7f8f6f394640 1 --2- 172.21.15.83:0/4245298134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f68100e90 0x7f8f681982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.765+0000 7f8f6f394640 1 -- 172.21.15.83:0/4245298134 >> 172.21.15.83:0/4245298134 conn(0x7f8f680fc400 msgr2=0x7f8f68069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:07.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.765+0000 7f8f6f394640 1 -- 172.21.15.83:0/4245298134 shutdown_connections 2024-04-25T11:41:07.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.765+0000 7f8f6f394640 1 -- 172.21.15.83:0/4245298134 wait complete. 2024-04-25T11:41:07.764 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:07.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.873+0000 7f8432387640 1 -- 172.21.15.83:0/663724052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f842c103190 msgr2=0x7f842c103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:07.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.873+0000 7f8432387640 1 --2- 172.21.15.83:0/663724052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f842c103190 0x7f842c103610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8420009900 tx=0x7f842002a410 comp rx=0 tx=0).stop 2024-04-25T11:41:07.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.873+0000 7f8432387640 1 -- 172.21.15.83:0/663724052 shutdown_connections 2024-04-25T11:41:07.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.873+0000 7f8432387640 1 --2- 172.21.15.83:0/663724052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f842c103190 0x7f842c103610 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.873+0000 7f8432387640 1 --2- 172.21.15.83:0/663724052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f842c103b50 0x7f842c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.873+0000 7f8432387640 1 --2- 172.21.15.83:0/663724052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f842c101fa0 0x7f842c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.873+0000 7f8432387640 1 -- 172.21.15.83:0/663724052 >> 172.21.15.83:0/663724052 conn(0x7f842c0fd550 msgr2=0x7f842c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:07.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.873+0000 7f8432387640 1 -- 172.21.15.83:0/663724052 shutdown_connections 2024-04-25T11:41:07.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.873+0000 7f8432387640 1 -- 172.21.15.83:0/663724052 wait complete. 2024-04-25T11:41:07.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.873+0000 7f8432387640 1 Processor -- start 2024-04-25T11:41:07.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.873+0000 7f8432387640 1 -- start start 2024-04-25T11:41:07.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8432387640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f842c101fa0 0x7f842c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:07.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8432387640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f842c103190 0x7f842c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:07.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8432387640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f842c103b50 0x7f842c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:07.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8432387640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f842c10dae0 con 0x7f842c103b50 2024-04-25T11:41:07.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8432387640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f842c10d960 con 0x7f842c103190 2024-04-25T11:41:07.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8432387640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f842c10dc60 con 0x7f842c101fa0 2024-04-25T11:41:07.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8430924640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f842c103b50 0x7f842c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8430924640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f842c103b50 0x7f842c19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53684/0 (socket says 172.21.15.83:53684) 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8430924640 1 -- 172.21.15.83:0/2706642389 learned_addr learned my addr 172.21.15.83:0/2706642389 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f842bfff640 1 --2- 172.21.15.83:0/2706642389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f842c101fa0 0x7f842c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8430924640 1 -- 172.21.15.83:0/2706642389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f842c101fa0 msgr2=0x7f842c1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8430924640 1 --2- 172.21.15.83:0/2706642389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f842c101fa0 0x7f842c1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8430924640 1 -- 172.21.15.83:0/2706642389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f842c103190 msgr2=0x7f842c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f842b7fe640 1 --2- 172.21.15.83:0/2706642389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f842c103190 0x7f842c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8430924640 1 --2- 172.21.15.83:0/2706642389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f842c103190 0x7f842c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8430924640 1 -- 172.21.15.83:0/2706642389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842c19fde0 con 0x7f842c103b50 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8430924640 1 --2- 172.21.15.83:0/2706642389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f842c103b50 0x7f842c19f770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f842400eb10 tx=0x7f842400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f84297fa640 1 -- 172.21.15.83:0/2706642389 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424004b20 con 0x7f842c103b50 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8432387640 1 -- 172.21.15.83:0/2706642389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842c1a00d0 con 0x7f842c103b50 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f84297fa640 1 -- 172.21.15.83:0/2706642389 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8424004cc0 con 0x7f842c103b50 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f84297fa640 1 -- 172.21.15.83:0/2706642389 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424010670 con 0x7f842c103b50 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.874+0000 7f8432387640 1 -- 172.21.15.83:0/2706642389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842c1a0620 con 0x7f842c103b50 2024-04-25T11:41:07.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.876+0000 7f8432387640 1 -- 172.21.15.83:0/2706642389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f842c109360 con 0x7f842c103b50 2024-04-25T11:41:07.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.877+0000 7f84297fa640 1 -- 172.21.15.83:0/2706642389 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8424005020 con 0x7f842c103b50 2024-04-25T11:41:07.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.877+0000 7f84297fa640 1 --2- 172.21.15.83:0/2706642389 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84140a3bb0 0x7f84140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:07.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.877+0000 7f842bfff640 1 --2- 172.21.15.83:0/2706642389 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84140a3bb0 0x7f84140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:07.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.877+0000 7f84297fa640 1 -- 172.21.15.83:0/2706642389 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f84240d0530 con 0x7f842c103b50 2024-04-25T11:41:07.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.877+0000 7f842bfff640 1 --2- 172.21.15.83:0/2706642389 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84140a3bb0 0x7f84140a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f842c102ff0 tx=0x7f841c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:07.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:07.882+0000 7f84297fa640 1 -- 172.21.15.83:0/2706642389 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f842409d3b0 con 0x7f842c103b50 2024-04-25T11:41:08.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:08.099+0000 7f8432387640 1 -- 172.21.15.83:0/2706642389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f842c062b50 con 0x7f842c103b50 2024-04-25T11:41:08.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:08.099+0000 7f84297fa640 1 -- 172.21.15.83:0/2706642389 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f842409ccb0 con 0x7f842c103b50 2024-04-25T11:41:08.087 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:08.087 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T11:41:08.087 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 1 2024-04-25T11:41:08.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:08.101+0000 7f8432387640 1 -- 172.21.15.83:0/2706642389 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84140a3bb0 msgr2=0x7f84140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:08.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:08.101+0000 7f8432387640 1 --2- 172.21.15.83:0/2706642389 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84140a3bb0 0x7f84140a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f842c102ff0 tx=0x7f841c00a2d0 comp rx=0 tx=0).stop 2024-04-25T11:41:08.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:08.101+0000 7f8432387640 1 -- 172.21.15.83:0/2706642389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f842c103b50 msgr2=0x7f842c19f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:08.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:08.101+0000 7f8432387640 1 --2- 172.21.15.83:0/2706642389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f842c103b50 0x7f842c19f770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f842400eb10 tx=0x7f842400c2d0 comp rx=0 tx=0).stop 2024-04-25T11:41:08.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:08.101+0000 7f8432387640 1 -- 172.21.15.83:0/2706642389 shutdown_connections 2024-04-25T11:41:08.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:08.101+0000 7f8432387640 1 --2- 172.21.15.83:0/2706642389 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84140a3bb0 0x7f84140a6070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:08.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:08.101+0000 7f8432387640 1 --2- 172.21.15.83:0/2706642389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f842c103b50 0x7f842c19f770 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:08.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:08.101+0000 7f8432387640 1 --2- 172.21.15.83:0/2706642389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f842c103190 0x7f842c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:08.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:08.101+0000 7f8432387640 1 --2- 172.21.15.83:0/2706642389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f842c101fa0 0x7f842c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:08.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:08.101+0000 7f8432387640 1 -- 172.21.15.83:0/2706642389 >> 172.21.15.83:0/2706642389 conn(0x7f842c0fd550 msgr2=0x7f842c106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:08.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:08.101+0000 7f8432387640 1 -- 172.21.15.83:0/2706642389 shutdown_connections 2024-04-25T11:41:08.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:08.101+0000 7f8432387640 1 -- 172.21.15.83:0/2706642389 wait complete. 2024-04-25T11:41:08.099 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:08 smithi083 ceph-mon[29330]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:08.099 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:08 smithi083 ceph-mon[29330]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l;count:12 2024-04-25T11:41:08.099 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:08 smithi083 ceph-mon[29330]: Deploying daemon mds.l on smithi111 2024-04-25T11:41:08.099 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:08 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/153170843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:08.099 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:08 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/4245298134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:41:08.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:08 smithi094 ceph-mon[33032]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:08.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:08 smithi094 ceph-mon[33032]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l;count:12 2024-04-25T11:41:08.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:08 smithi094 ceph-mon[33032]: Deploying daemon mds.l on smithi111 2024-04-25T11:41:08.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:08 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/153170843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:08.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:08 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/4245298134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:41:08.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:08 smithi111 ceph-mon[32784]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:08.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:08 smithi111 ceph-mon[32784]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l;count:12 2024-04-25T11:41:08.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:08 smithi111 ceph-mon[32784]: Deploying daemon mds.l on smithi111 2024-04-25T11:41:08.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:08 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/153170843' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:08.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:08 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/4245298134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:41:09.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:09 smithi083 ceph-mon[29330]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:09.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:09 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2706642389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:09.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:09 smithi094 ceph-mon[33032]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:09.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:09 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2706642389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:09.293 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:09 smithi111 ceph-mon[32784]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:09.293 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:09 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2706642389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:09.293 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:41:09 smithi111 systemd[1]: Starting Ceph mds.l for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:41:09.557 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:41:09 smithi111 podman[62872]: 2024-04-25 11:41:09.209334845 +0000 UTC m=+0.017990434 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:41:09.557 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:41:09 smithi111 podman[62872]: 2024-04-25 11:41:09.314826865 +0000 UTC m=+0.123482442 container create ee8cf3d49253b6dfbe6acbc0500c7acd733196cc2e03c2f287cafec673fc269b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:41:09.557 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:41:09 smithi111 podman[62872]: 2024-04-25 11:41:09.545328292 +0000 UTC m=+0.353983873 container init ee8cf3d49253b6dfbe6acbc0500c7acd733196cc2e03c2f287cafec673fc269b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T11:41:09.557 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:41:09 smithi111 podman[62872]: 2024-04-25 11:41:09.547827012 +0000 UTC m=+0.356482591 container start ee8cf3d49253b6dfbe6acbc0500c7acd733196cc2e03c2f287cafec673fc269b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-25T11:41:09.821 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:41:09 smithi111 ceph-mds[62891]: -- 172.21.15.111:0/1442870113 <== mon.1 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56434f109a40 con 0x56434f0dd000 2024-04-25T11:41:09.821 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:41:09 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-l[62887]: starting mds.l at 2024-04-25T11:41:09.822 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:41:09 smithi111 bash[62872]: ee8cf3d49253b6dfbe6acbc0500c7acd733196cc2e03c2f287cafec673fc269b 2024-04-25T11:41:09.822 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:41:09 smithi111 systemd[1]: Started Ceph mds.l for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:41:10.101 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.222+0000 7f523dc05640 1 -- 172.21.15.83:0/3787733455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5238102a70 msgr2=0x7f5238104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.222+0000 7f523dc05640 1 --2- 172.21.15.83:0/3787733455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5238102a70 0x7f5238104e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5228009900 tx=0x7f522802a410 comp rx=0 tx=0).stop 2024-04-25T11:41:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.222+0000 7f523dc05640 1 -- 172.21.15.83:0/3787733455 shutdown_connections 2024-04-25T11:41:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.222+0000 7f523dc05640 1 --2- 172.21.15.83:0/3787733455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5238107cd0 0x7f523810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.222+0000 7f523dc05640 1 --2- 172.21.15.83:0/3787733455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52381053a0 0x7f5238107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.222+0000 7f523dc05640 1 --2- 172.21.15.83:0/3787733455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5238102a70 0x7f5238104e60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.222+0000 7f523dc05640 1 -- 172.21.15.83:0/3787733455 >> 172.21.15.83:0/3787733455 conn(0x7f52380fc400 msgr2=0x7f52380fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.222+0000 7f523dc05640 1 -- 172.21.15.83:0/3787733455 shutdown_connections 2024-04-25T11:41:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.222+0000 7f523dc05640 1 -- 172.21.15.83:0/3787733455 wait complete. 2024-04-25T11:41:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.222+0000 7f523dc05640 1 Processor -- start 2024-04-25T11:41:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.222+0000 7f523dc05640 1 -- start start 2024-04-25T11:41:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f523dc05640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5238102a70 0x7f5238198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f523dc05640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52381053a0 0x7f5238198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:10.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f523dc05640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5238107cd0 0x7f523819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f523dc05640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f523810dad0 con 0x7f5238102a70 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f523dc05640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f523810d950 con 0x7f52381053a0 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f523dc05640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f523810dc50 con 0x7f5238107cd0 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f52377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5238102a70 0x7f5238198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f52377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5238102a70 0x7f5238198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53702/0 (socket says 172.21.15.83:53702) 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f52377fe640 1 -- 172.21.15.83:0/783711566 learned_addr learned my addr 172.21.15.83:0/783711566 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f522ffff640 1 --2- 172.21.15.83:0/783711566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52381053a0 0x7f5238198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f5237fff640 1 --2- 172.21.15.83:0/783711566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5238107cd0 0x7f523819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f52377fe640 1 -- 172.21.15.83:0/783711566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5238107cd0 msgr2=0x7f523819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f52377fe640 1 --2- 172.21.15.83:0/783711566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5238107cd0 0x7f523819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f52377fe640 1 -- 172.21.15.83:0/783711566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52381053a0 msgr2=0x7f5238198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f52377fe640 1 --2- 172.21.15.83:0/783711566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52381053a0 0x7f5238198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f52377fe640 1 -- 172.21.15.83:0/783711566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523819fe70 con 0x7f5238102a70 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f52377fe640 1 --2- 172.21.15.83:0/783711566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5238102a70 0x7f5238198250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5228005940 tx=0x7f5228031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f52357fa640 1 -- 172.21.15.83:0/783711566 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522803f030 con 0x7f5238102a70 2024-04-25T11:41:10.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f523dc05640 1 -- 172.21.15.83:0/783711566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52381a0100 con 0x7f5238102a70 2024-04-25T11:41:10.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f52357fa640 1 -- 172.21.15.83:0/783711566 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5228003aa0 con 0x7f5238102a70 2024-04-25T11:41:10.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f52357fa640 1 -- 172.21.15.83:0/783711566 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5228035070 con 0x7f5238102a70 2024-04-25T11:41:10.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.223+0000 7f523dc05640 1 -- 172.21.15.83:0/783711566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52381a05f0 con 0x7f5238102a70 2024-04-25T11:41:10.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.224+0000 7f523dc05640 1 -- 172.21.15.83:0/783711566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52380fdef0 con 0x7f5238102a70 2024-04-25T11:41:10.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.225+0000 7f52357fa640 1 -- 172.21.15.83:0/783711566 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5228042030 con 0x7f5238102a70 2024-04-25T11:41:10.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.226+0000 7f52357fa640 1 --2- 172.21.15.83:0/783711566 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f52240a3bb0 0x7f52240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:10.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.226+0000 7f522ffff640 1 --2- 172.21.15.83:0/783711566 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f52240a3bb0 0x7f52240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:10.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.226+0000 7f52357fa640 1 -- 172.21.15.83:0/783711566 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f52280f04f0 con 0x7f5238102a70 2024-04-25T11:41:10.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.226+0000 7f522ffff640 1 --2- 172.21.15.83:0/783711566 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f52240a3bb0 0x7f52240a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5238199760 tx=0x7f522000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:10.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.231+0000 7f52357fa640 1 -- 172.21.15.83:0/783711566 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52280bcd20 con 0x7f5238102a70 2024-04-25T11:41:10.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.447+0000 7f523dc05640 1 -- 172.21.15.83:0/783711566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f523802d5c0 con 0x7f5238102a70 2024-04-25T11:41:10.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.447+0000 7f52357fa640 1 -- 172.21.15.83:0/783711566 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f52280c1020 con 0x7f5238102a70 2024-04-25T11:41:10.435 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:10.435 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T11:41:10.436 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 2 2024-04-25T11:41:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.449+0000 7f523dc05640 1 -- 172.21.15.83:0/783711566 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f52240a3bb0 msgr2=0x7f52240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:10.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.449+0000 7f523dc05640 1 --2- 172.21.15.83:0/783711566 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f52240a3bb0 0x7f52240a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5238199760 tx=0x7f522000b390 comp rx=0 tx=0).stop 2024-04-25T11:41:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.449+0000 7f523dc05640 1 -- 172.21.15.83:0/783711566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5238102a70 msgr2=0x7f5238198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.449+0000 7f523dc05640 1 --2- 172.21.15.83:0/783711566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5238102a70 0x7f5238198250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5228005940 tx=0x7f5228031a50 comp rx=0 tx=0).stop 2024-04-25T11:41:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.449+0000 7f523dc05640 1 -- 172.21.15.83:0/783711566 shutdown_connections 2024-04-25T11:41:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.449+0000 7f523dc05640 1 --2- 172.21.15.83:0/783711566 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f52240a3bb0 0x7f52240a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.449+0000 7f523dc05640 1 --2- 172.21.15.83:0/783711566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5238107cd0 0x7f523819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.449+0000 7f523dc05640 1 --2- 172.21.15.83:0/783711566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52381053a0 0x7f5238198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.449+0000 7f523dc05640 1 --2- 172.21.15.83:0/783711566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5238102a70 0x7f5238198250 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.450+0000 7f523dc05640 1 -- 172.21.15.83:0/783711566 >> 172.21.15.83:0/783711566 conn(0x7f52380fc400 msgr2=0x7f5238104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.450+0000 7f523dc05640 1 -- 172.21.15.83:0/783711566 shutdown_connections 2024-04-25T11:41:10.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:10.450+0000 7f523dc05640 1 -- 172.21.15.83:0/783711566 wait complete. 2024-04-25T11:41:10.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:10 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.111:6832/2234401967,v1:172.21.15.111:6833/2234401967] up:boot 2024-04-25T11:41:10.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:10 smithi111 ceph-mon[32784]: fsmap 1 up:standby 2024-04-25T11:41:10.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T11:41:10.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:10.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:10.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:10.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:10.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:10.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:10 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:10.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:10 smithi111 ceph-mon[32784]: Deploying daemon mds.k on smithi094 2024-04-25T11:41:10.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:10 smithi111 ceph-mon[32784]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:10.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:10 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/783711566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:10.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:10 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.111:6832/2234401967,v1:172.21.15.111:6833/2234401967] up:boot 2024-04-25T11:41:10.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:10 smithi083 ceph-mon[29330]: fsmap 1 up:standby 2024-04-25T11:41:10.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T11:41:10.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:10.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:10.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:10.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:10.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:10.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:10 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:10.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:10 smithi083 ceph-mon[29330]: Deploying daemon mds.k on smithi094 2024-04-25T11:41:10.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:10 smithi083 ceph-mon[29330]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:10.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:10 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/783711566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:11.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:10 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.111:6832/2234401967,v1:172.21.15.111:6833/2234401967] up:boot 2024-04-25T11:41:11.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:10 smithi094 ceph-mon[33032]: fsmap 1 up:standby 2024-04-25T11:41:11.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T11:41:11.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:11.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:11.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:11.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:11.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:11.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:10 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:11.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:10 smithi094 ceph-mon[33032]: Deploying daemon mds.k on smithi094 2024-04-25T11:41:11.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:10 smithi094 ceph-mon[33032]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:11.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:10 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/783711566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:12.023 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:41:11 smithi094 systemd[1]: Starting Ceph mds.k for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:41:12.449 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:12.456 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:41:12 smithi094 podman[62314]: 2024-04-25 11:41:12.200318743 +0000 UTC m=+0.111754159 container create be554274b8aea69ed308fc9ea926c5678e17cd68abfbdc9d37d5672604693fdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:41:12.457 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:41:12 smithi094 podman[62314]: 2024-04-25 11:41:12.106932865 +0000 UTC m=+0.018368292 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:41:12.457 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:41:12 smithi094 podman[62314]: 2024-04-25 11:41:12.444678019 +0000 UTC m=+0.356113427 container init be554274b8aea69ed308fc9ea926c5678e17cd68abfbdc9d37d5672604693fdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T11:41:12.457 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:41:12 smithi094 podman[62314]: 2024-04-25 11:41:12.447234892 +0000 UTC m=+0.358670316 container start be554274b8aea69ed308fc9ea926c5678e17cd68abfbdc9d37d5672604693fdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T11:41:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.573+0000 7fb3c6364640 1 -- 172.21.15.83:0/234432168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c0105340 msgr2=0x7fb3c0107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.573+0000 7fb3c6364640 1 --2- 172.21.15.83:0/234432168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c0105340 0x7fb3c0107730 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009900 tx=0x7fb3a802a410 comp rx=0 tx=0).stop 2024-04-25T11:41:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.573+0000 7fb3c6364640 1 -- 172.21.15.83:0/234432168 shutdown_connections 2024-04-25T11:41:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.573+0000 7fb3c6364640 1 --2- 172.21.15.83:0/234432168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c0105340 0x7fb3c0107730 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.573+0000 7fb3c6364640 1 --2- 172.21.15.83:0/234432168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3c0107c70 0x7fb3c010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.573+0000 7fb3c6364640 1 --2- 172.21.15.83:0/234432168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c0102a10 0x7fb3c0104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.573+0000 7fb3c6364640 1 -- 172.21.15.83:0/234432168 >> 172.21.15.83:0/234432168 conn(0x7fb3c00fc400 msgr2=0x7fb3c00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.573+0000 7fb3c6364640 1 -- 172.21.15.83:0/234432168 shutdown_connections 2024-04-25T11:41:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.573+0000 7fb3c6364640 1 -- 172.21.15.83:0/234432168 wait complete. 2024-04-25T11:41:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.573+0000 7fb3c6364640 1 Processor -- start 2024-04-25T11:41:12.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.573+0000 7fb3c6364640 1 -- start start 2024-04-25T11:41:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3c6364640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3c0102a10 0x7fb3c0198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3c6364640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c0105340 0x7fb3c01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3c6364640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c0107c70 0x7fb3c019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3c6364640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c010daa0 con 0x7fb3c0105340 2024-04-25T11:41:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3c6364640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c010d920 con 0x7fb3c0107c70 2024-04-25T11:41:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3c6364640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c010dc20 con 0x7fb3c0102a10 2024-04-25T11:41:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3bf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c0105340 0x7fb3c01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3bf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c0105340 0x7fb3c01986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35294/0 (socket says 172.21.15.83:35294) 2024-04-25T11:41:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3bf7fe640 1 -- 172.21.15.83:0/3252142749 learned_addr learned my addr 172.21.15.83:0/3252142749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3bffff640 1 --2- 172.21.15.83:0/3252142749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3c0102a10 0x7fb3c0198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3c4901640 1 --2- 172.21.15.83:0/3252142749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c0107c70 0x7fb3c019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:12.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3bf7fe640 1 -- 172.21.15.83:0/3252142749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3c0102a10 msgr2=0x7fb3c0198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:12.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3bf7fe640 1 --2- 172.21.15.83:0/3252142749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3c0102a10 0x7fb3c0198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3bf7fe640 1 -- 172.21.15.83:0/3252142749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c0107c70 msgr2=0x7fb3c019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:12.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3bf7fe640 1 --2- 172.21.15.83:0/3252142749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c0107c70 0x7fb3c019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3bf7fe640 1 -- 172.21.15.83:0/3252142749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c019fda0 con 0x7fb3c0105340 2024-04-25T11:41:12.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.574+0000 7fb3bf7fe640 1 --2- 172.21.15.83:0/3252142749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c0105340 0x7fb3c01986c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8006010 tx=0x7fb3a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:12.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.576+0000 7fb3bd7fa640 1 -- 172.21.15.83:0/3252142749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a803f030 con 0x7fb3c0105340 2024-04-25T11:41:12.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.577+0000 7fb3c6364640 1 -- 172.21.15.83:0/3252142749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c01a0030 con 0x7fb3c0105340 2024-04-25T11:41:12.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.577+0000 7fb3bd7fa640 1 -- 172.21.15.83:0/3252142749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3a8003cb0 con 0x7fb3c0105340 2024-04-25T11:41:12.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.577+0000 7fb3c6364640 1 -- 172.21.15.83:0/3252142749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c01a0570 con 0x7fb3c0105340 2024-04-25T11:41:12.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.577+0000 7fb3bd7fa640 1 -- 172.21.15.83:0/3252142749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8035070 con 0x7fb3c0105340 2024-04-25T11:41:12.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.578+0000 7fb3c6364640 1 -- 172.21.15.83:0/3252142749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a00052c0 con 0x7fb3c0105340 2024-04-25T11:41:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.579+0000 7fb3bd7fa640 1 -- 172.21.15.83:0/3252142749 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb3a8042030 con 0x7fb3c0105340 2024-04-25T11:41:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.580+0000 7fb3bd7fa640 1 --2- 172.21.15.83:0/3252142749 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb3ac0a3c30 0x7fb3ac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:12.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.580+0000 7fb3bffff640 1 --2- 172.21.15.83:0/3252142749 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb3ac0a3c30 0x7fb3ac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:12.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.580+0000 7fb3bd7fa640 1 -- 172.21.15.83:0/3252142749 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb3a80f0940 con 0x7fb3c0105340 2024-04-25T11:41:12.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.580+0000 7fb3bffff640 1 --2- 172.21.15.83:0/3252142749 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb3ac0a3c30 0x7fb3ac0a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0005d00 tx=0x7fb3b000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:12.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.584+0000 7fb3bd7fa640 1 -- 172.21.15.83:0/3252142749 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb3a8003e60 con 0x7fb3c0105340 2024-04-25T11:41:12.772 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:41:12 smithi094 ceph-mds[62333]: -- 172.21.15.94:0/1018182726 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f543849a40 con 0x55f54381cc00 2024-04-25T11:41:12.772 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:41:12 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-k[62329]: starting mds.k at 2024-04-25T11:41:12.772 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:41:12 smithi094 bash[62314]: be554274b8aea69ed308fc9ea926c5678e17cd68abfbdc9d37d5672604693fdc 2024-04-25T11:41:12.772 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:41:12 smithi094 systemd[1]: Started Ceph mds.k for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:41:12.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.821+0000 7fb3c6364640 1 -- 172.21.15.83:0/3252142749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3a00055b0 con 0x7fb3c0105340 2024-04-25T11:41:12.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.822+0000 7fb3bd7fa640 1 -- 172.21.15.83:0/3252142749 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fb3a80bd170 con 0x7fb3c0105340 2024-04-25T11:41:12.808 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:12.809 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T11:41:12.809 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 2 2024-04-25T11:41:12.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.824+0000 7fb3c6364640 1 -- 172.21.15.83:0/3252142749 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb3ac0a3c30 msgr2=0x7fb3ac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.824+0000 7fb3c6364640 1 --2- 172.21.15.83:0/3252142749 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb3ac0a3c30 0x7fb3ac0a60f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0005d00 tx=0x7fb3b000b410 comp rx=0 tx=0).stop 2024-04-25T11:41:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.824+0000 7fb3c6364640 1 -- 172.21.15.83:0/3252142749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c0105340 msgr2=0x7fb3c01986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.824+0000 7fb3c6364640 1 --2- 172.21.15.83:0/3252142749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c0105340 0x7fb3c01986c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8006010 tx=0x7fb3a80042f0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.824+0000 7fb3c6364640 1 -- 172.21.15.83:0/3252142749 shutdown_connections 2024-04-25T11:41:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.824+0000 7fb3c6364640 1 --2- 172.21.15.83:0/3252142749 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb3ac0a3c30 0x7fb3ac0a60f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.824+0000 7fb3c6364640 1 --2- 172.21.15.83:0/3252142749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb3c0105340 0x7fb3c01986c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.824+0000 7fb3c6364640 1 --2- 172.21.15.83:0/3252142749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3c0107c70 0x7fb3c019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.824+0000 7fb3c6364640 1 --2- 172.21.15.83:0/3252142749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3c0102a10 0x7fb3c0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.824+0000 7fb3c6364640 1 -- 172.21.15.83:0/3252142749 >> 172.21.15.83:0/3252142749 conn(0x7fb3c00fc400 msgr2=0x7fb3c0103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.824+0000 7fb3c6364640 1 -- 172.21.15.83:0/3252142749 shutdown_connections 2024-04-25T11:41:12.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:12.824+0000 7fb3c6364640 1 -- 172.21.15.83:0/3252142749 wait complete. 2024-04-25T11:41:13.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:12 smithi083 ceph-mon[29330]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:13.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:13.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:13.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:13.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:13.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:13.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:12 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:13.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:12 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3252142749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:13.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:12 smithi094 ceph-mon[33032]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:13.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:13.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:13.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:13.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:13.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:13.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:12 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:13.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:12 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3252142749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:13.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:12 smithi111 ceph-mon[32784]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:13.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:13.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:13.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:13.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:13.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:13.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:12 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:13.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:12 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3252142749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:14.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:13 smithi083 ceph-mon[29330]: Deploying daemon mds.j on smithi083 2024-04-25T11:41:14.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:13 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.94:6832/3250237836,v1:172.21.15.94:6833/3250237836] up:boot 2024-04-25T11:41:14.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:13 smithi083 ceph-mon[29330]: fsmap 2 up:standby 2024-04-25T11:41:14.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:13 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T11:41:14.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:13 smithi094 ceph-mon[33032]: Deploying daemon mds.j on smithi083 2024-04-25T11:41:14.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:13 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.94:6832/3250237836,v1:172.21.15.94:6833/3250237836] up:boot 2024-04-25T11:41:14.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:13 smithi094 ceph-mon[33032]: fsmap 2 up:standby 2024-04-25T11:41:14.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:13 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T11:41:14.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:13 smithi111 ceph-mon[32784]: Deploying daemon mds.j on smithi083 2024-04-25T11:41:14.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:13 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.94:6832/3250237836,v1:172.21.15.94:6833/3250237836] up:boot 2024-04-25T11:41:14.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:13 smithi111 ceph-mon[32784]: fsmap 2 up:standby 2024-04-25T11:41:14.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:13 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T11:41:14.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:14.848 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 25 11:41:14 smithi083 systemd[1]: Starting Ceph mds.j for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:41:14.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.955+0000 7f320c3e2640 1 -- 172.21.15.83:0/1065313844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204101fc0 msgr2=0x7f32041023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:14.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.955+0000 7f320c3e2640 1 --2- 172.21.15.83:0/1065313844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204101fc0 0x7f32041023e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3200009900 tx=0x7f320002a410 comp rx=0 tx=0).stop 2024-04-25T11:41:14.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.955+0000 7f320c3e2640 1 -- 172.21.15.83:0/1065313844 shutdown_connections 2024-04-25T11:41:14.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.955+0000 7f320c3e2640 1 --2- 172.21.15.83:0/1065313844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3204103b70 0x7f320410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.955+0000 7f320c3e2640 1 --2- 172.21.15.83:0/1065313844 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32041031b0 0x7f3204103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.955+0000 7f320c3e2640 1 --2- 172.21.15.83:0/1065313844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204101fc0 0x7f32041023e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.955+0000 7f320c3e2640 1 -- 172.21.15.83:0/1065313844 >> 172.21.15.83:0/1065313844 conn(0x7f32040fd530 msgr2=0x7f32040ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:14.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.955+0000 7f320c3e2640 1 -- 172.21.15.83:0/1065313844 shutdown_connections 2024-04-25T11:41:14.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.955+0000 7f320c3e2640 1 -- 172.21.15.83:0/1065313844 wait complete. 2024-04-25T11:41:14.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320c3e2640 1 Processor -- start 2024-04-25T11:41:14.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320c3e2640 1 -- start start 2024-04-25T11:41:14.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320c3e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204101fc0 0x7f32041981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320c3e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32041031b0 0x7f3204198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320c3e2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3204103b70 0x7f320419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320c3e2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f320410db00 con 0x7f3204101fc0 2024-04-25T11:41:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320c3e2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f320410d980 con 0x7f32041031b0 2024-04-25T11:41:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320c3e2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f320410dc80 con 0x7f3204103b70 2024-04-25T11:41:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320a17e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204101fc0 0x7f32041981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320a17e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204101fc0 0x7f32041981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35308/0 (socket says 172.21.15.83:35308) 2024-04-25T11:41:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320a17e640 1 -- 172.21.15.83:0/2106643714 learned_addr learned my addr 172.21.15.83:0/2106643714 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320997d640 1 --2- 172.21.15.83:0/2106643714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32041031b0 0x7f3204198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320a17e640 1 -- 172.21.15.83:0/2106643714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3204103b70 msgr2=0x7f320419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320a97f640 1 --2- 172.21.15.83:0/2106643714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3204103b70 0x7f320419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320a17e640 1 --2- 172.21.15.83:0/2106643714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3204103b70 0x7f320419f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320a17e640 1 -- 172.21.15.83:0/2106643714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32041031b0 msgr2=0x7f3204198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320a17e640 1 --2- 172.21.15.83:0/2106643714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32041031b0 0x7f3204198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320a17e640 1 -- 172.21.15.83:0/2106643714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320419fe10 con 0x7f3204101fc0 2024-04-25T11:41:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.956+0000 7f320997d640 1 --2- 172.21.15.83:0/2106643714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32041031b0 0x7f3204198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.957+0000 7f320a17e640 1 --2- 172.21.15.83:0/2106643714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204101fc0 0x7f32041981f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3200006010 tx=0x7f32000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.957+0000 7f31fb7fe640 1 -- 172.21.15.83:0/2106643714 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320003f030 con 0x7f3204101fc0 2024-04-25T11:41:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.957+0000 7f31fb7fe640 1 -- 172.21.15.83:0/2106643714 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3200003cb0 con 0x7f3204101fc0 2024-04-25T11:41:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.957+0000 7f320c3e2640 1 -- 172.21.15.83:0/2106643714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32041a00a0 con 0x7f3204101fc0 2024-04-25T11:41:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.957+0000 7f31fb7fe640 1 -- 172.21.15.83:0/2106643714 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200035070 con 0x7f3204101fc0 2024-04-25T11:41:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.957+0000 7f320c3e2640 1 -- 172.21.15.83:0/2106643714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32041a05e0 con 0x7f3204101fc0 2024-04-25T11:41:14.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.958+0000 7f320c3e2640 1 -- 172.21.15.83:0/2106643714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32041093b0 con 0x7f3204101fc0 2024-04-25T11:41:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.959+0000 7f31fb7fe640 1 -- 172.21.15.83:0/2106643714 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3200042030 con 0x7f3204101fc0 2024-04-25T11:41:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.959+0000 7f31fb7fe640 1 --2- 172.21.15.83:0/2106643714 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f31f00a3c30 0x7f31f00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.959+0000 7f320997d640 1 --2- 172.21.15.83:0/2106643714 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f31f00a3c30 0x7f31f00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.960+0000 7f31fb7fe640 1 -- 172.21.15.83:0/2106643714 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f32000f05f0 con 0x7f3204101fc0 2024-04-25T11:41:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.960+0000 7f320997d640 1 --2- 172.21.15.83:0/2106643714 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f31f00a3c30 0x7f31f00a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3204199700 tx=0x7f31f4005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:14.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:14.964+0000 7f31fb7fe640 1 -- 172.21.15.83:0/2106643714 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f32000bd450 con 0x7f3204101fc0 2024-04-25T11:41:15.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:14 smithi094 ceph-mon[33032]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:15.112 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:14 smithi083 ceph-mon[29330]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:15.112 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 25 11:41:15 smithi083 podman[70376]: 2024-04-25 11:41:14.911387794 +0000 UTC m=+0.025609263 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:41:15.112 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 25 11:41:15 smithi083 podman[70376]: 2024-04-25 11:41:15.011297542 +0000 UTC m=+0.125519015 container create fb95b994b5c6c369fca2d4e8c0949f03d1c82deb3455d8865269dd86f3dc02b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T11:41:15.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:15.195+0000 7f320c3e2640 1 -- 172.21.15.83:0/2106643714 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3204062b50 con 0x7f3204101fc0 2024-04-25T11:41:15.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:15.196+0000 7f31fb7fe640 1 -- 172.21.15.83:0/2106643714 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f32000bcd50 con 0x7f3204101fc0 2024-04-25T11:41:15.183 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:15.183 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T11:41:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 3 2024-04-25T11:41:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:15.198+0000 7f320c3e2640 1 -- 172.21.15.83:0/2106643714 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f31f00a3c30 msgr2=0x7f31f00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:15.198+0000 7f320c3e2640 1 --2- 172.21.15.83:0/2106643714 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f31f00a3c30 0x7f31f00a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3204199700 tx=0x7f31f4005ec0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:15.198+0000 7f320c3e2640 1 -- 172.21.15.83:0/2106643714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204101fc0 msgr2=0x7f32041981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:15.198+0000 7f320c3e2640 1 --2- 172.21.15.83:0/2106643714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204101fc0 0x7f32041981f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3200006010 tx=0x7f32000042f0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:15.198+0000 7f320c3e2640 1 -- 172.21.15.83:0/2106643714 shutdown_connections 2024-04-25T11:41:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:15.198+0000 7f320c3e2640 1 --2- 172.21.15.83:0/2106643714 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f31f00a3c30 0x7f31f00a60f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:15.198+0000 7f320c3e2640 1 --2- 172.21.15.83:0/2106643714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3204103b70 0x7f320419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:15.198+0000 7f320c3e2640 1 --2- 172.21.15.83:0/2106643714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32041031b0 0x7f3204198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:15.198+0000 7f320c3e2640 1 --2- 172.21.15.83:0/2106643714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3204101fc0 0x7f32041981f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:15.198+0000 7f320c3e2640 1 -- 172.21.15.83:0/2106643714 >> 172.21.15.83:0/2106643714 conn(0x7f32040fd530 msgr2=0x7f3204106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:15.198+0000 7f320c3e2640 1 -- 172.21.15.83:0/2106643714 shutdown_connections 2024-04-25T11:41:15.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:15.198+0000 7f320c3e2640 1 -- 172.21.15.83:0/2106643714 wait complete. 2024-04-25T11:41:15.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:14 smithi111 ceph-mon[32784]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:15.655 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 25 11:41:15 smithi083 podman[70376]: 2024-04-25 11:41:15.239302478 +0000 UTC m=+0.353523957 container init fb95b994b5c6c369fca2d4e8c0949f03d1c82deb3455d8865269dd86f3dc02b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T11:41:15.655 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 25 11:41:15 smithi083 podman[70376]: 2024-04-25 11:41:15.242066594 +0000 UTC m=+0.356288064 container start fb95b994b5c6c369fca2d4e8c0949f03d1c82deb3455d8865269dd86f3dc02b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-25T11:41:15.655 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 25 11:41:15 smithi083 ceph-mds[70416]: -- 172.21.15.83:0/1121808147 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5607ec221a40 con 0x5607ec1f4c00 2024-04-25T11:41:15.655 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 25 11:41:15 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-j[70410]: starting mds.j at 2024-04-25T11:41:15.656 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 25 11:41:15 smithi083 bash[70376]: fb95b994b5c6c369fca2d4e8c0949f03d1c82deb3455d8865269dd86f3dc02b6 2024-04-25T11:41:15.656 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 25 11:41:15 smithi083 systemd[1]: Started Ceph mds.j for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:41:16.228 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:15 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2106643714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:16.228 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:15 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:16.228 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:15 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:16.228 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:15 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:16.228 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:15 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:16.228 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:15 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:16.229 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:15 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:16.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:15 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2106643714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:16.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:15 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:16.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:15 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:16.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:15 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:16.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:15 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:16.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:15 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:16.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:15 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:16.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:15 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2106643714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:16.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:15 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:16.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:15 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:16.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:15 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:16.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:15 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:16.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:15 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:16.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:15 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:17.196 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:17.218 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:16 smithi083 ceph-mon[29330]: Deploying daemon mds.d on smithi083 2024-04-25T11:41:17.218 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:16 smithi083 ceph-mon[29330]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:17.218 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:16 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.83:6834/3911246904,v1:172.21.15.83:6835/3911246904] up:boot 2024-04-25T11:41:17.218 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:16 smithi083 ceph-mon[29330]: fsmap 3 up:standby 2024-04-25T11:41:17.219 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:16 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T11:41:17.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:16 smithi094 ceph-mon[33032]: Deploying daemon mds.d on smithi083 2024-04-25T11:41:17.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:16 smithi094 ceph-mon[33032]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:17.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:16 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.83:6834/3911246904,v1:172.21.15.83:6835/3911246904] up:boot 2024-04-25T11:41:17.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:16 smithi094 ceph-mon[33032]: fsmap 3 up:standby 2024-04-25T11:41:17.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:16 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T11:41:17.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:16 smithi111 ceph-mon[32784]: Deploying daemon mds.d on smithi083 2024-04-25T11:41:17.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:16 smithi111 ceph-mon[32784]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:17.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:16 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.83:6834/3911246904,v1:172.21.15.83:6835/3911246904] up:boot 2024-04-25T11:41:17.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:16 smithi111 ceph-mon[32784]: fsmap 3 up:standby 2024-04-25T11:41:17.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:16 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T11:41:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.337+0000 7f299ee35640 1 -- 172.21.15.83:0/942468773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2998102ab0 msgr2=0x7f2998104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.337+0000 7f299ee35640 1 --2- 172.21.15.83:0/942468773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2998102ab0 0x7f2998104ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f298c009900 tx=0x7f298c02a410 comp rx=0 tx=0).stop 2024-04-25T11:41:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.337+0000 7f299ee35640 1 -- 172.21.15.83:0/942468773 shutdown_connections 2024-04-25T11:41:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.337+0000 7f299ee35640 1 --2- 172.21.15.83:0/942468773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2998107d10 0x7f299810a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.337+0000 7f299ee35640 1 --2- 172.21.15.83:0/942468773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29981053e0 0x7f29981077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.337+0000 7f299ee35640 1 --2- 172.21.15.83:0/942468773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2998102ab0 0x7f2998104ea0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.337+0000 7f299ee35640 1 -- 172.21.15.83:0/942468773 >> 172.21.15.83:0/942468773 conn(0x7f29980fc400 msgr2=0x7f29980fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.338+0000 7f299ee35640 1 -- 172.21.15.83:0/942468773 shutdown_connections 2024-04-25T11:41:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.338+0000 7f299ee35640 1 -- 172.21.15.83:0/942468773 wait complete. 2024-04-25T11:41:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.338+0000 7f299ee35640 1 Processor -- start 2024-04-25T11:41:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.338+0000 7f299ee35640 1 -- start start 2024-04-25T11:41:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.338+0000 7f299ee35640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2998102ab0 0x7f2998102420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.338+0000 7f299ee35640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29981053e0 0x7f2998100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.338+0000 7f299ee35640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2998107d10 0x7f2998100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.338+0000 7f299ee35640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f299810daf0 con 0x7f2998107d10 2024-04-25T11:41:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.338+0000 7f299ee35640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f299810d970 con 0x7f2998102ab0 2024-04-25T11:41:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.338+0000 7f299ee35640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f299810dc70 con 0x7f29981053e0 2024-04-25T11:41:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f299d3d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2998107d10 0x7f2998100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f299d3d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2998107d10 0x7f2998100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35346/0 (socket says 172.21.15.83:35346) 2024-04-25T11:41:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f299d3d2640 1 -- 172.21.15.83:0/1242939474 learned_addr learned my addr 172.21.15.83:0/1242939474 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f2997fff640 1 --2- 172.21.15.83:0/1242939474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29981053e0 0x7f2998100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f299d3d2640 1 -- 172.21.15.83:0/1242939474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29981053e0 msgr2=0x7f2998100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f299d3d2640 1 --2- 172.21.15.83:0/1242939474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29981053e0 0x7f2998100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f299cbd1640 1 --2- 172.21.15.83:0/1242939474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2998102ab0 0x7f2998102420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f299d3d2640 1 -- 172.21.15.83:0/1242939474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2998102ab0 msgr2=0x7f2998102420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f299d3d2640 1 --2- 172.21.15.83:0/1242939474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2998102ab0 0x7f2998102420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f299d3d2640 1 -- 172.21.15.83:0/1242939474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2998101830 con 0x7f2998107d10 2024-04-25T11:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f299d3d2640 1 --2- 172.21.15.83:0/1242939474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2998107d10 0x7f2998100fa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f299000cce0 tx=0x7f299000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f2995ffb640 1 -- 172.21.15.83:0/1242939474 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299000f450 con 0x7f2998107d10 2024-04-25T11:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f2995ffb640 1 -- 172.21.15.83:0/1242939474 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f299000faa0 con 0x7f2998107d10 2024-04-25T11:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f2995ffb640 1 -- 172.21.15.83:0/1242939474 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990018b70 con 0x7f2998107d10 2024-04-25T11:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f299ee35640 1 -- 172.21.15.83:0/1242939474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29981a22d0 con 0x7f2998107d10 2024-04-25T11:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.339+0000 7f299ee35640 1 -- 172.21.15.83:0/1242939474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29981a27d0 con 0x7f2998107d10 2024-04-25T11:41:17.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.340+0000 7f299ee35640 1 -- 172.21.15.83:0/1242939474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29980fdef0 con 0x7f2998107d10 2024-04-25T11:41:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.341+0000 7f2995ffb640 1 -- 172.21.15.83:0/1242939474 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2990018d10 con 0x7f2998107d10 2024-04-25T11:41:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.342+0000 7f2995ffb640 1 --2- 172.21.15.83:0/1242939474 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f29880a3bb0 0x7f29880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.342+0000 7f299cbd1640 1 --2- 172.21.15.83:0/1242939474 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f29880a3bb0 0x7f29880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.342+0000 7f2995ffb640 1 -- 172.21.15.83:0/1242939474 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f29900d04d0 con 0x7f2998107d10 2024-04-25T11:41:17.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.342+0000 7f299cbd1640 1 --2- 172.21.15.83:0/1242939474 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f29880a3bb0 0x7f29880a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f298c005c40 tx=0x7f298c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:17.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.347+0000 7f2995ffb640 1 -- 172.21.15.83:0/1242939474 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2990014030 con 0x7f2998107d10 2024-04-25T11:41:17.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.593+0000 7f299ee35640 1 -- 172.21.15.83:0/1242939474 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2998062b50 con 0x7f2998107d10 2024-04-25T11:41:17.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.593+0000 7f2995ffb640 1 -- 172.21.15.83:0/1242939474 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f299009d350 con 0x7f2998107d10 2024-04-25T11:41:17.580 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:17.580 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T11:41:17.581 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 4 2024-04-25T11:41:17.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.596+0000 7f299ee35640 1 -- 172.21.15.83:0/1242939474 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f29880a3bb0 msgr2=0x7f29880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:17.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.596+0000 7f299ee35640 1 --2- 172.21.15.83:0/1242939474 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f29880a3bb0 0x7f29880a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f298c005c40 tx=0x7f298c000bc0 comp rx=0 tx=0).stop 2024-04-25T11:41:17.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.596+0000 7f299ee35640 1 -- 172.21.15.83:0/1242939474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2998107d10 msgr2=0x7f2998100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:17.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.596+0000 7f299ee35640 1 --2- 172.21.15.83:0/1242939474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2998107d10 0x7f2998100fa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f299000cce0 tx=0x7f299000a620 comp rx=0 tx=0).stop 2024-04-25T11:41:17.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.596+0000 7f299ee35640 1 -- 172.21.15.83:0/1242939474 shutdown_connections 2024-04-25T11:41:17.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.596+0000 7f299ee35640 1 --2- 172.21.15.83:0/1242939474 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f29880a3bb0 0x7f29880a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:17.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.596+0000 7f299ee35640 1 --2- 172.21.15.83:0/1242939474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2998107d10 0x7f2998100fa0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:17.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.596+0000 7f299ee35640 1 --2- 172.21.15.83:0/1242939474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29981053e0 0x7f2998100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:17.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.596+0000 7f299ee35640 1 --2- 172.21.15.83:0/1242939474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2998102ab0 0x7f2998102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:17.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.596+0000 7f299ee35640 1 -- 172.21.15.83:0/1242939474 >> 172.21.15.83:0/1242939474 conn(0x7f29980fc400 msgr2=0x7f2998103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:17.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.596+0000 7f299ee35640 1 -- 172.21.15.83:0/1242939474 shutdown_connections 2024-04-25T11:41:17.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:17.596+0000 7f299ee35640 1 -- 172.21.15.83:0/1242939474 wait complete. 2024-04-25T11:41:17.593 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 25 11:41:17 smithi083 systemd[1]: Starting Ceph mds.d for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:41:17.905 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 25 11:41:17 smithi083 podman[70746]: 2024-04-25 11:41:17.593094172 +0000 UTC m=+0.018506089 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:41:17.906 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 25 11:41:17 smithi083 podman[70746]: 2024-04-25 11:41:17.716951399 +0000 UTC m=+0.142363314 container create 5b29031497ff8b7f075f31363679ccd5a2b394f2c01e0b5ad110a8effad95416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T11:41:18.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:17 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1242939474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:18.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:17 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1242939474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:18.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:17 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1242939474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:18.405 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 25 11:41:17 smithi083 podman[70746]: 2024-04-25 11:41:17.945980133 +0000 UTC m=+0.371392045 container init 5b29031497ff8b7f075f31363679ccd5a2b394f2c01e0b5ad110a8effad95416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:41:18.405 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 25 11:41:17 smithi083 podman[70746]: 2024-04-25 11:41:17.948793184 +0000 UTC m=+0.374205097 container start 5b29031497ff8b7f075f31363679ccd5a2b394f2c01e0b5ad110a8effad95416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-d, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T11:41:18.405 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 25 11:41:17 smithi083 ceph-mds[70767]: -- 172.21.15.83:0/4230692205 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5593d6bc1a40 con 0x5593d6b94c00 2024-04-25T11:41:18.406 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 25 11:41:17 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-d[70763]: starting mds.d at 2024-04-25T11:41:18.406 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 25 11:41:18 smithi083 bash[70746]: 5b29031497ff8b7f075f31363679ccd5a2b394f2c01e0b5ad110a8effad95416 2024-04-25T11:41:18.406 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 25 11:41:18 smithi083 systemd[1]: Started Ceph mds.d for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:41:19.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:18 smithi094 ceph-mon[33032]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:19.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:19.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:19.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:19.008 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:19.009 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:19.009 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:19.009 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:18 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.83:6836/3006959432,v1:172.21.15.83:6837/3006959432] up:boot 2024-04-25T11:41:19.009 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:18 smithi094 ceph-mon[33032]: fsmap 4 up:standby 2024-04-25T11:41:19.009 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:18 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T11:41:19.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:18 smithi111 ceph-mon[32784]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:19.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:19.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:19.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:19.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:19.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:19.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:19.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:18 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.83:6836/3006959432,v1:172.21.15.83:6837/3006959432] up:boot 2024-04-25T11:41:19.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:18 smithi111 ceph-mon[32784]: fsmap 4 up:standby 2024-04-25T11:41:19.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:18 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T11:41:19.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:18 smithi083 ceph-mon[29330]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:19.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:19.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:19.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:19.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:19.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:19.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:19.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:18 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.83:6836/3006959432,v1:172.21.15.83:6837/3006959432] up:boot 2024-04-25T11:41:19.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:18 smithi083 ceph-mon[29330]: fsmap 4 up:standby 2024-04-25T11:41:19.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:18 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T11:41:19.595 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.720+0000 7fa70b344640 1 -- 172.21.15.83:0/3200190834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa704103190 msgr2=0x7fa704105600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.720+0000 7fa70b344640 1 --2- 172.21.15.83:0/3200190834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa704103190 0x7fa704105600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009900 tx=0x7fa6f402a410 comp rx=0 tx=0).stop 2024-04-25T11:41:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.720+0000 7fa70b344640 1 -- 172.21.15.83:0/3200190834 shutdown_connections 2024-04-25T11:41:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.720+0000 7fa70b344640 1 --2- 172.21.15.83:0/3200190834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa704103190 0x7fa704105600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.720+0000 7fa70b344640 1 --2- 172.21.15.83:0/3200190834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa704105b40 0x7fa704105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.720+0000 7fa70b344640 1 --2- 172.21.15.83:0/3200190834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa704100820 0x7fa704102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.720+0000 7fa70b344640 1 -- 172.21.15.83:0/3200190834 >> 172.21.15.83:0/3200190834 conn(0x7fa7040fc440 msgr2=0x7fa7040fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.720+0000 7fa70b344640 1 -- 172.21.15.83:0/3200190834 shutdown_connections 2024-04-25T11:41:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.720+0000 7fa70b344640 1 -- 172.21.15.83:0/3200190834 wait complete. 2024-04-25T11:41:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa70b344640 1 Processor -- start 2024-04-25T11:41:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa70b344640 1 -- start start 2024-04-25T11:41:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa70b344640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa704100820 0x7fa70419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa70b344640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa704103190 0x7fa70419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa70b344640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa704105b40 0x7fa7041a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa70b344640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041140e0 con 0x7fa704100820 2024-04-25T11:41:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa70b344640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa704113f60 con 0x7fa704105b40 2024-04-25T11:41:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa70b344640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa704114260 con 0x7fa704103190 2024-04-25T11:41:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa7090e0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa704100820 0x7fa70419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa7090e0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa704100820 0x7fa70419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35374/0 (socket says 172.21.15.83:35374) 2024-04-25T11:41:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa7090e0640 1 -- 172.21.15.83:0/1500986811 learned_addr learned my addr 172.21.15.83:0/1500986811 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa7098e1640 1 --2- 172.21.15.83:0/1500986811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa704105b40 0x7fa7041a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa7088df640 1 --2- 172.21.15.83:0/1500986811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa704103190 0x7fa70419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa7090e0640 1 -- 172.21.15.83:0/1500986811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa704103190 msgr2=0x7fa70419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa7090e0640 1 --2- 172.21.15.83:0/1500986811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa704103190 0x7fa70419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa7090e0640 1 -- 172.21.15.83:0/1500986811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa704105b40 msgr2=0x7fa7041a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa7090e0640 1 --2- 172.21.15.83:0/1500986811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa704105b40 0x7fa7041a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.721+0000 7fa7090e0640 1 -- 172.21.15.83:0/1500986811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7041a6400 con 0x7fa704100820 2024-04-25T11:41:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.722+0000 7fa7090e0640 1 --2- 172.21.15.83:0/1500986811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa704100820 0x7fa70419e7e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa70000d770 tx=0x7fa70000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.722+0000 7fa6fa7fc640 1 -- 172.21.15.83:0/1500986811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700017070 con 0x7fa704100820 2024-04-25T11:41:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.722+0000 7fa6fa7fc640 1 -- 172.21.15.83:0/1500986811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7000054d0 con 0x7fa704100820 2024-04-25T11:41:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.722+0000 7fa70b344640 1 -- 172.21.15.83:0/1500986811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7041a66f0 con 0x7fa704100820 2024-04-25T11:41:19.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.722+0000 7fa6fa7fc640 1 -- 172.21.15.83:0/1500986811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700004c30 con 0x7fa704100820 2024-04-25T11:41:19.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.722+0000 7fa70b344640 1 -- 172.21.15.83:0/1500986811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7041a6c40 con 0x7fa704100820 2024-04-25T11:41:19.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.723+0000 7fa70b344640 1 -- 172.21.15.83:0/1500986811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7040fdf30 con 0x7fa704100820 2024-04-25T11:41:19.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.724+0000 7fa6fa7fc640 1 -- 172.21.15.83:0/1500986811 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa700004e10 con 0x7fa704100820 2024-04-25T11:41:19.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.725+0000 7fa6fa7fc640 1 --2- 172.21.15.83:0/1500986811 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa6f00a3c30 0x7fa6f00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:19.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.725+0000 7fa7088df640 1 --2- 172.21.15.83:0/1500986811 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa6f00a3c30 0x7fa6f00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:19.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.725+0000 7fa6fa7fc640 1 -- 172.21.15.83:0/1500986811 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa7000cf9a0 con 0x7fa704100820 2024-04-25T11:41:19.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.725+0000 7fa7088df640 1 --2- 172.21.15.83:0/1500986811 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa6f00a3c30 0x7fa6f00a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa70419fcf0 tx=0x7fa6f40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:19.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.729+0000 7fa6fa7fc640 1 -- 172.21.15.83:0/1500986811 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa70009c820 con 0x7fa704100820 2024-04-25T11:41:19.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.944+0000 7fa70b344640 1 -- 172.21.15.83:0/1500986811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa70402d5c0 con 0x7fa704100820 2024-04-25T11:41:19.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.945+0000 7fa6fa7fc640 1 -- 172.21.15.83:0/1500986811 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7fa70009c120 con 0x7fa704100820 2024-04-25T11:41:19.932 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:19.932 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T11:41:19.932 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 5 2024-04-25T11:41:19.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.947+0000 7fa70b344640 1 -- 172.21.15.83:0/1500986811 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa6f00a3c30 msgr2=0x7fa6f00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:19.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.947+0000 7fa70b344640 1 --2- 172.21.15.83:0/1500986811 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa6f00a3c30 0x7fa6f00a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa70419fcf0 tx=0x7fa6f40314a0 comp rx=0 tx=0).stop 2024-04-25T11:41:19.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.947+0000 7fa70b344640 1 -- 172.21.15.83:0/1500986811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa704100820 msgr2=0x7fa70419e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:19.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.947+0000 7fa70b344640 1 --2- 172.21.15.83:0/1500986811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa704100820 0x7fa70419e7e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa70000d770 tx=0x7fa70000dc30 comp rx=0 tx=0).stop 2024-04-25T11:41:19.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.947+0000 7fa70b344640 1 -- 172.21.15.83:0/1500986811 shutdown_connections 2024-04-25T11:41:19.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.947+0000 7fa70b344640 1 --2- 172.21.15.83:0/1500986811 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa6f00a3c30 0x7fa6f00a60f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:19.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.947+0000 7fa70b344640 1 --2- 172.21.15.83:0/1500986811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa704105b40 0x7fa7041a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:19.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.947+0000 7fa70b344640 1 --2- 172.21.15.83:0/1500986811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa704103190 0x7fa70419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:19.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.947+0000 7fa70b344640 1 --2- 172.21.15.83:0/1500986811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa704100820 0x7fa70419e7e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:19.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.947+0000 7fa70b344640 1 -- 172.21.15.83:0/1500986811 >> 172.21.15.83:0/1500986811 conn(0x7fa7040fc440 msgr2=0x7fa704101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:19.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.947+0000 7fa70b344640 1 -- 172.21.15.83:0/1500986811 shutdown_connections 2024-04-25T11:41:19.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:19.947+0000 7fa70b344640 1 -- 172.21.15.83:0/1500986811 wait complete. 2024-04-25T11:41:20.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:19 smithi094 ceph-mon[33032]: Deploying daemon mds.e on smithi094 2024-04-25T11:41:20.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:19 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1500986811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:20.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:19 smithi111 ceph-mon[32784]: Deploying daemon mds.e on smithi094 2024-04-25T11:41:20.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:19 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1500986811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:20.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:19 smithi083 ceph-mon[29330]: Deploying daemon mds.e on smithi094 2024-04-25T11:41:20.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:19 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1500986811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:20.438 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:41:20 smithi094 systemd[1]: Starting Ceph mds.e for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:41:20.689 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:41:20 smithi094 podman[62638]: 2024-04-25 11:41:20.354643321 +0000 UTC m=+0.018130763 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:41:20.690 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:41:20 smithi094 podman[62638]: 2024-04-25 11:41:20.472934548 +0000 UTC m=+0.136421971 container create 5e04a085005824a06e992c92cb7260d786e5aa08213de0bcf6c1bcbaef1dbd0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:41:20.690 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:41:20 smithi094 podman[62638]: 2024-04-25 11:41:20.700939273 +0000 UTC m=+0.364426696 container init 5e04a085005824a06e992c92cb7260d786e5aa08213de0bcf6c1bcbaef1dbd0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-e, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T11:41:20.690 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:41:20 smithi094 podman[62638]: 2024-04-25 11:41:20.703485276 +0000 UTC m=+0.366972702 container start 5e04a085005824a06e992c92cb7260d786e5aa08213de0bcf6c1bcbaef1dbd0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T11:41:20.984 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:41:20 smithi094 ceph-mds[62657]: -- 172.21.15.94:0/328588103 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559fff5e1a40 con 0x559fff5b5000 2024-04-25T11:41:20.984 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:41:20 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-e[62653]: starting mds.e at 2024-04-25T11:41:20.984 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:41:20 smithi094 bash[62638]: 5e04a085005824a06e992c92cb7260d786e5aa08213de0bcf6c1bcbaef1dbd0e 2024-04-25T11:41:20.984 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:41:20 smithi094 systemd[1]: Started Ceph mds.e for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:41:21.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:20 smithi094 ceph-mon[33032]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:21.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:21.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:21.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:21.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:21.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:21.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:20 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:21.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:20 smithi083 ceph-mon[29330]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:21.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:21.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:21.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:21.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:21.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:21.307 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:20 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:21.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:20 smithi111 ceph-mon[32784]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:21.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:21.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:21.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:21.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:21.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:21.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:20 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:21.947 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:22.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.062+0000 7f0991dcd640 1 -- 172.21.15.83:0/1309730730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c100e90 msgr2=0x7f098c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.062+0000 7f0991dcd640 1 --2- 172.21.15.83:0/1309730730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c100e90 0x7f098c1012b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f097c009900 tx=0x7f097c02a410 comp rx=0 tx=0).stop 2024-04-25T11:41:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.062+0000 7f0991dcd640 1 -- 172.21.15.83:0/1309730730 shutdown_connections 2024-04-25T11:41:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.062+0000 7f0991dcd640 1 --2- 172.21.15.83:0/1309730730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c1029a0 0x7f098c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.062+0000 7f0991dcd640 1 --2- 172.21.15.83:0/1309730730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f098c101fe0 0x7f098c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.062+0000 7f0991dcd640 1 --2- 172.21.15.83:0/1309730730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c100e90 0x7f098c1012b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.062+0000 7f0991dcd640 1 -- 172.21.15.83:0/1309730730 >> 172.21.15.83:0/1309730730 conn(0x7f098c0fc400 msgr2=0x7f098c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.062+0000 7f0991dcd640 1 -- 172.21.15.83:0/1309730730 shutdown_connections 2024-04-25T11:41:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.062+0000 7f0991dcd640 1 -- 172.21.15.83:0/1309730730 wait complete. 2024-04-25T11:41:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.063+0000 7f0991dcd640 1 Processor -- start 2024-04-25T11:41:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.063+0000 7f0991dcd640 1 -- start start 2024-04-25T11:41:22.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.063+0000 7f0991dcd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f098c100e90 0x7f098c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.063+0000 7f0991dcd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c101fe0 0x7f098c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.063+0000 7f0991dcd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c1029a0 0x7f098c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.063+0000 7f0991dcd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f098c10c9c0 con 0x7f098c101fe0 2024-04-25T11:41:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.063+0000 7f0991dcd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f098c10c840 con 0x7f098c1029a0 2024-04-25T11:41:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.063+0000 7f0991dcd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f098c10cb40 con 0x7f098c100e90 2024-04-25T11:41:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.063+0000 7f0983fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c101fe0 0x7f098c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.063+0000 7f0983fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c101fe0 0x7f098c198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35386/0 (socket says 172.21.15.83:35386) 2024-04-25T11:41:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.063+0000 7f0983fff640 1 -- 172.21.15.83:0/3673032903 learned_addr learned my addr 172.21.15.83:0/3673032903 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.063+0000 7f098bfff640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c1029a0 0x7f098c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.063+0000 7f098b7fe640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f098c100e90 0x7f098c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.064+0000 7f0983fff640 1 -- 172.21.15.83:0/3673032903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f098c100e90 msgr2=0x7f098c198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.064+0000 7f0983fff640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f098c100e90 0x7f098c198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.064+0000 7f0983fff640 1 -- 172.21.15.83:0/3673032903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c1029a0 msgr2=0x7f098c19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.064+0000 7f0983fff640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c1029a0 0x7f098c19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.064+0000 7f0983fff640 1 -- 172.21.15.83:0/3673032903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098c19fe60 con 0x7f098c101fe0 2024-04-25T11:41:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.064+0000 7f098b7fe640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f098c100e90 0x7f098c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.064+0000 7f098bfff640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c1029a0 0x7f098c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.064+0000 7f0983fff640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c101fe0 0x7f098c198780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f097400d770 tx=0x7f097400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.064+0000 7f09897fa640 1 -- 172.21.15.83:0/3673032903 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974017070 con 0x7f098c101fe0 2024-04-25T11:41:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.064+0000 7f0991dcd640 1 -- 172.21.15.83:0/3673032903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f098c1a0150 con 0x7f098c101fe0 2024-04-25T11:41:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.064+0000 7f09897fa640 1 -- 172.21.15.83:0/3673032903 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0974009c30 con 0x7f098c101fe0 2024-04-25T11:41:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.064+0000 7f09897fa640 1 -- 172.21.15.83:0/3673032903 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974003cc0 con 0x7f098c101fe0 2024-04-25T11:41:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.064+0000 7f0991dcd640 1 -- 172.21.15.83:0/3673032903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f098c1a06f0 con 0x7f098c101fe0 2024-04-25T11:41:22.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.065+0000 7f0991dcd640 1 -- 172.21.15.83:0/3673032903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f098c1a0950 con 0x7f098c101fe0 2024-04-25T11:41:22.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.066+0000 7f09897fa640 1 -- 172.21.15.83:0/3673032903 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0974003e60 con 0x7f098c101fe0 2024-04-25T11:41:22.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.067+0000 7f09897fa640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f09780a3bb0 0x7f09780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:22.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.067+0000 7f098b7fe640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f09780a3bb0 0x7f09780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:22.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.067+0000 7f09897fa640 1 -- 172.21.15.83:0/3673032903 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f09740cf9c0 con 0x7f098c101fe0 2024-04-25T11:41:22.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.067+0000 7f098b7fe640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f09780a3bb0 0x7f09780a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f097c000c00 tx=0x7f097c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:22.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.075+0000 7f09897fa640 1 -- 172.21.15.83:0/3673032903 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f097409c820 con 0x7f098c101fe0 2024-04-25T11:41:22.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:22 smithi094 ceph-mon[33032]: Deploying daemon mds.f on smithi111 2024-04-25T11:41:22.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:22 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.94:6834/4086957377,v1:172.21.15.94:6835/4086957377] up:boot 2024-04-25T11:41:22.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:22 smithi094 ceph-mon[33032]: fsmap 5 up:standby 2024-04-25T11:41:22.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:22 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T11:41:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.299+0000 7f0991dcd640 1 -- 172.21.15.83:0/3673032903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f098c1a0b20 con 0x7f098c101fe0 2024-04-25T11:41:22.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.300+0000 7f09897fa640 1 -- 172.21.15.83:0/3673032903 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7f0974009de0 con 0x7f098c101fe0 2024-04-25T11:41:22.287 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:22.287 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T11:41:22.287 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 6 2024-04-25T11:41:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.302+0000 7f0991dcd640 1 -- 172.21.15.83:0/3673032903 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f09780a3bb0 msgr2=0x7f09780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.302+0000 7f0991dcd640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f09780a3bb0 0x7f09780a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f097c000c00 tx=0x7f097c0314a0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.302+0000 7f0991dcd640 1 -- 172.21.15.83:0/3673032903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c101fe0 msgr2=0x7f098c198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.302+0000 7f0991dcd640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c101fe0 0x7f098c198780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f097400d770 tx=0x7f097400dc30 comp rx=0 tx=0).stop 2024-04-25T11:41:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.302+0000 7f0991dcd640 1 -- 172.21.15.83:0/3673032903 shutdown_connections 2024-04-25T11:41:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.302+0000 7f0991dcd640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f09780a3bb0 0x7f09780a6070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.302+0000 7f0991dcd640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f098c101fe0 0x7f098c198780 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.302+0000 7f0991dcd640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098c1029a0 0x7f098c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.302+0000 7f0991dcd640 1 --2- 172.21.15.83:0/3673032903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f098c100e90 0x7f098c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:22.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.302+0000 7f0991dcd640 1 -- 172.21.15.83:0/3673032903 >> 172.21.15.83:0/3673032903 conn(0x7f098c0fc400 msgr2=0x7f098c069190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:22.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.302+0000 7f0991dcd640 1 -- 172.21.15.83:0/3673032903 shutdown_connections 2024-04-25T11:41:22.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:22.302+0000 7f0991dcd640 1 -- 172.21.15.83:0/3673032903 wait complete. 2024-04-25T11:41:22.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:22 smithi083 ceph-mon[29330]: Deploying daemon mds.f on smithi111 2024-04-25T11:41:22.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:22 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.94:6834/4086957377,v1:172.21.15.94:6835/4086957377] up:boot 2024-04-25T11:41:22.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:22 smithi083 ceph-mon[29330]: fsmap 5 up:standby 2024-04-25T11:41:22.299 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:22 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T11:41:22.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:22 smithi111 ceph-mon[32784]: Deploying daemon mds.f on smithi111 2024-04-25T11:41:22.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:22 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.94:6834/4086957377,v1:172.21.15.94:6835/4086957377] up:boot 2024-04-25T11:41:22.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:22 smithi111 ceph-mon[32784]: fsmap 5 up:standby 2024-04-25T11:41:22.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:22 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T11:41:22.994 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:41:22 smithi111 systemd[1]: Starting Ceph mds.f for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:41:23.244 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:23 smithi111 ceph-mon[32784]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:23.245 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:23 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3673032903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:23.245 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:41:23 smithi111 podman[63176]: 2024-04-25 11:41:23.043299008 +0000 UTC m=+0.018221135 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:41:23.245 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:41:23 smithi111 podman[63176]: 2024-04-25 11:41:23.153654386 +0000 UTC m=+0.128576513 container create 16d4d3696cf6bf1b3b0ff072a515be54aab2eaabd606c3ea9709576479f1dd72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T11:41:23.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:23 smithi094 ceph-mon[33032]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:23.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:23 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3673032903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:23.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:23 smithi083 ceph-mon[29330]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:23.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:23 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3673032903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:23.821 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:41:23 smithi111 podman[63176]: 2024-04-25 11:41:23.389533037 +0000 UTC m=+0.364455164 container init 16d4d3696cf6bf1b3b0ff072a515be54aab2eaabd606c3ea9709576479f1dd72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-f, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T11:41:23.822 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:41:23 smithi111 podman[63176]: 2024-04-25 11:41:23.392262663 +0000 UTC m=+0.367184793 container start 16d4d3696cf6bf1b3b0ff072a515be54aab2eaabd606c3ea9709576479f1dd72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T11:41:23.822 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:41:23 smithi111 ceph-mds[63195]: -- 172.21.15.111:0/2307268222 <== mon.1 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b7e76ada40 con 0x55b7e7681000 2024-04-25T11:41:23.822 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:41:23 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-f[63191]: starting mds.f at 2024-04-25T11:41:23.822 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:41:23 smithi111 bash[63176]: 16d4d3696cf6bf1b3b0ff072a515be54aab2eaabd606c3ea9709576479f1dd72 2024-04-25T11:41:23.822 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:41:23 smithi111 systemd[1]: Started Ceph mds.f for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:41:24.300 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:24.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.425+0000 7fb9934c2640 1 -- 172.21.15.83:0/2973776401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c102a20 msgr2=0x7fb98c104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:24.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.425+0000 7fb9934c2640 1 --2- 172.21.15.83:0/2973776401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c102a20 0x7fb98c104e10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb988009900 tx=0x7fb98802a410 comp rx=0 tx=0).stop 2024-04-25T11:41:24.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.425+0000 7fb9934c2640 1 -- 172.21.15.83:0/2973776401 shutdown_connections 2024-04-25T11:41:24.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.425+0000 7fb9934c2640 1 --2- 172.21.15.83:0/2973776401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb98c107c80 0x7fb98c10a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.425+0000 7fb9934c2640 1 --2- 172.21.15.83:0/2973776401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb98c105350 0x7fb98c107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.425+0000 7fb9934c2640 1 --2- 172.21.15.83:0/2973776401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c102a20 0x7fb98c104e10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.425+0000 7fb9934c2640 1 -- 172.21.15.83:0/2973776401 >> 172.21.15.83:0/2973776401 conn(0x7fb98c0fc430 msgr2=0x7fb98c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:24.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.425+0000 7fb9934c2640 1 -- 172.21.15.83:0/2973776401 shutdown_connections 2024-04-25T11:41:24.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.425+0000 7fb9934c2640 1 -- 172.21.15.83:0/2973776401 wait complete. 2024-04-25T11:41:24.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb9934c2640 1 Processor -- start 2024-04-25T11:41:24.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb9934c2640 1 -- start start 2024-04-25T11:41:24.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb9934c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb98c102a20 0x7fb98c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb9934c2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb98c105350 0x7fb98c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb9934c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c107c80 0x7fb98c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb9934c2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c10dae0 con 0x7fb98c107c80 2024-04-25T11:41:24.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb9934c2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c10d960 con 0x7fb98c102a20 2024-04-25T11:41:24.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb9934c2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c10dc60 con 0x7fb98c105350 2024-04-25T11:41:24.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb991a5f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c107c80 0x7fb98c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb991a5f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c107c80 0x7fb98c19f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46838/0 (socket says 172.21.15.83:46838) 2024-04-25T11:41:24.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb991a5f640 1 -- 172.21.15.83:0/2892178085 learned_addr learned my addr 172.21.15.83:0/2892178085 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:24.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb990a5d640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb98c105350 0x7fb98c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb99125e640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb98c102a20 0x7fb98c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb991a5f640 1 -- 172.21.15.83:0/2892178085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb98c105350 msgr2=0x7fb98c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:24.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb991a5f640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb98c105350 0x7fb98c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb991a5f640 1 -- 172.21.15.83:0/2892178085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb98c102a20 msgr2=0x7fb98c198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:24.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb991a5f640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb98c102a20 0x7fb98c198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb991a5f640 1 -- 172.21.15.83:0/2892178085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c19fe70 con 0x7fb98c107c80 2024-04-25T11:41:24.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb990a5d640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb98c105350 0x7fb98c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:24.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb99125e640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb98c102a20 0x7fb98c198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:24.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb991a5f640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c107c80 0x7fb98c19f800 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb98400eaa0 tx=0x7fb98400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:24.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb9827fc640 1 -- 172.21.15.83:0/2892178085 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98400cdb0 con 0x7fb98c107c80 2024-04-25T11:41:24.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb9827fc640 1 -- 172.21.15.83:0/2892178085 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb984005020 con 0x7fb98c107c80 2024-04-25T11:41:24.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.426+0000 7fb9934c2640 1 -- 172.21.15.83:0/2892178085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98c1a0160 con 0x7fb98c107c80 2024-04-25T11:41:24.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.427+0000 7fb9827fc640 1 -- 172.21.15.83:0/2892178085 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9840116d0 con 0x7fb98c107c80 2024-04-25T11:41:24.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.427+0000 7fb9934c2640 1 -- 172.21.15.83:0/2892178085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98c1a0660 con 0x7fb98c107c80 2024-04-25T11:41:24.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.427+0000 7fb9934c2640 1 -- 172.21.15.83:0/2892178085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98c0fdf20 con 0x7fb98c107c80 2024-04-25T11:41:24.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.429+0000 7fb9827fc640 1 -- 172.21.15.83:0/2892178085 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb9840051d0 con 0x7fb98c107c80 2024-04-25T11:41:24.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.429+0000 7fb9827fc640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb9780a3bb0 0x7fb9780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:24.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.429+0000 7fb99125e640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb9780a3bb0 0x7fb9780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:24.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.430+0000 7fb9827fc640 1 -- 172.21.15.83:0/2892178085 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb9840d0600 con 0x7fb98c107c80 2024-04-25T11:41:24.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.430+0000 7fb99125e640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb9780a3bb0 0x7fb9780a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb988006010 tx=0x7fb988000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:24.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.433+0000 7fb9827fc640 1 -- 172.21.15.83:0/2892178085 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb984016030 con 0x7fb98c107c80 2024-04-25T11:41:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.648+0000 7fb9934c2640 1 -- 172.21.15.83:0/2892178085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb98c02d5c0 con 0x7fb98c107c80 2024-04-25T11:41:24.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.649+0000 7fb9827fc640 1 -- 172.21.15.83:0/2892178085 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5524 (secure 0 0 0) 0x7fb98409d480 con 0x7fb98c107c80 2024-04-25T11:41:24.635 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:24.636 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T11:41:24.636 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-04-25T11:41:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.651+0000 7fb9934c2640 1 -- 172.21.15.83:0/2892178085 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb9780a3bb0 msgr2=0x7fb9780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.651+0000 7fb9934c2640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb9780a3bb0 0x7fb9780a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb988006010 tx=0x7fb988000bc0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.651+0000 7fb9934c2640 1 -- 172.21.15.83:0/2892178085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c107c80 msgr2=0x7fb98c19f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.651+0000 7fb9934c2640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c107c80 0x7fb98c19f800 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb98400eaa0 tx=0x7fb98400ef60 comp rx=0 tx=0).stop 2024-04-25T11:41:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.651+0000 7fb9934c2640 1 -- 172.21.15.83:0/2892178085 shutdown_connections 2024-04-25T11:41:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.651+0000 7fb9934c2640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb9780a3bb0 0x7fb9780a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.651+0000 7fb9934c2640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb98c107c80 0x7fb98c19f800 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.651+0000 7fb9934c2640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb98c105350 0x7fb98c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.651+0000 7fb9934c2640 1 --2- 172.21.15.83:0/2892178085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb98c102a20 0x7fb98c198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.651+0000 7fb9934c2640 1 -- 172.21.15.83:0/2892178085 >> 172.21.15.83:0/2892178085 conn(0x7fb98c0fc430 msgr2=0x7fb98c103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.651+0000 7fb9934c2640 1 -- 172.21.15.83:0/2892178085 shutdown_connections 2024-04-25T11:41:24.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:24.651+0000 7fb9934c2640 1 -- 172.21.15.83:0/2892178085 wait complete. 2024-04-25T11:41:24.647 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:24 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:24.647 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:24 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:24.647 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:24 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:24.647 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:24 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:24.647 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:24 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:24.647 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:24 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:24.648 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:24 smithi083 ceph-mon[29330]: Deploying daemon mds.c on smithi111 2024-04-25T11:41:24.648 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:24 smithi083 ceph-mon[29330]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:24.648 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:24 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] up:boot 2024-04-25T11:41:24.648 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:24 smithi083 ceph-mon[29330]: fsmap 6 up:standby 2024-04-25T11:41:24.648 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:24 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T11:41:24.811 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:24 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:24.811 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:24 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:24.811 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:24 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:24.811 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:24 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:24.811 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:24 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:24.811 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:24 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:24.811 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:24 smithi094 ceph-mon[33032]: Deploying daemon mds.c on smithi111 2024-04-25T11:41:24.811 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:24 smithi094 ceph-mon[33032]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:24.811 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:24 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] up:boot 2024-04-25T11:41:24.811 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:24 smithi094 ceph-mon[33032]: fsmap 6 up:standby 2024-04-25T11:41:24.811 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:24 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T11:41:24.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:24 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:24.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:24 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:24.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:24 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:24.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:24 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:24.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:24 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:24.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:24 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:24.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:24 smithi111 ceph-mon[32784]: Deploying daemon mds.c on smithi111 2024-04-25T11:41:24.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:24 smithi111 ceph-mon[32784]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:24.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:24 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] up:boot 2024-04-25T11:41:24.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:24 smithi111 ceph-mon[32784]: fsmap 6 up:standby 2024-04-25T11:41:24.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:24 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T11:41:25.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:25 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2892178085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:25.822 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:41:25 smithi111 systemd[1]: Starting Ceph mds.c for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:41:25.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:25 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2892178085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:26.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:25 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2892178085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:26.212 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:41:25 smithi111 podman[63484]: 2024-04-25 11:41:25.826462692 +0000 UTC m=+0.018406895 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:41:26.212 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:41:25 smithi111 podman[63484]: 2024-04-25 11:41:25.931883896 +0000 UTC m=+0.123828145 container create b3c289675af19f5c42f96219dafb90e4828c44566e695d8d717180d6da7f990c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T11:41:26.212 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:41:26 smithi111 podman[63484]: 2024-04-25 11:41:26.142421566 +0000 UTC m=+0.334365752 container init b3c289675af19f5c42f96219dafb90e4828c44566e695d8d717180d6da7f990c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T11:41:26.212 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:41:26 smithi111 podman[63484]: 2024-04-25 11:41:26.145084978 +0000 UTC m=+0.337029173 container start b3c289675af19f5c42f96219dafb90e4828c44566e695d8d717180d6da7f990c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-c, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T11:41:26.212 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:41:26 smithi111 ceph-mds[63503]: -- 172.21.15.111:0/3352886009 <== mon.1 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b25d8e5a40 con 0x55b25d8b8800 2024-04-25T11:41:26.212 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:41:26 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-c[63499]: starting mds.c at 2024-04-25T11:41:26.212 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:41:26 smithi111 bash[63484]: b3c289675af19f5c42f96219dafb90e4828c44566e695d8d717180d6da7f990c 2024-04-25T11:41:26.545 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:41:26 smithi111 systemd[1]: Started Ceph mds.c for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:41:26.650 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:26.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.773+0000 7f9eb3472640 1 -- 172.21.15.83:0/4287096741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eac1029b0 msgr2=0x7f9eac1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:26.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.773+0000 7f9eb3472640 1 --2- 172.21.15.83:0/4287096741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eac1029b0 0x7f9eac1092d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4009ca0 tx=0x7f9ea402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:41:26.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.773+0000 7f9eb3472640 1 -- 172.21.15.83:0/4287096741 shutdown_connections 2024-04-25T11:41:26.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.773+0000 7f9eb3472640 1 --2- 172.21.15.83:0/4287096741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eac1029b0 0x7f9eac1092d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:26.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.773+0000 7f9eb3472640 1 --2- 172.21.15.83:0/4287096741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9eac101ff0 0x7f9eac102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:26.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.773+0000 7f9eb3472640 1 --2- 172.21.15.83:0/4287096741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eac100ea0 0x7f9eac1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:26.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.773+0000 7f9eb3472640 1 -- 172.21.15.83:0/4287096741 >> 172.21.15.83:0/4287096741 conn(0x7f9eac0fc430 msgr2=0x7f9eac0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:26.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.773+0000 7f9eb3472640 1 -- 172.21.15.83:0/4287096741 shutdown_connections 2024-04-25T11:41:26.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.773+0000 7f9eb3472640 1 -- 172.21.15.83:0/4287096741 wait complete. 2024-04-25T11:41:26.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.773+0000 7f9eb3472640 1 Processor -- start 2024-04-25T11:41:26.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.773+0000 7f9eb3472640 1 -- start start 2024-04-25T11:41:26.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb3472640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eac100ea0 0x7f9eac197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb3472640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9eac101ff0 0x7f9eac1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb3472640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eac1029b0 0x7f9eac19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb3472640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac10c9e0 con 0x7f9eac100ea0 2024-04-25T11:41:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb3472640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac10c860 con 0x7f9eac1029b0 2024-04-25T11:41:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb3472640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac10cb60 con 0x7f9eac101ff0 2024-04-25T11:41:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb120e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eac100ea0 0x7f9eac197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb120e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eac100ea0 0x7f9eac197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46846/0 (socket says 172.21.15.83:46846) 2024-04-25T11:41:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb120e640 1 -- 172.21.15.83:0/2094536708 learned_addr learned my addr 172.21.15.83:0/2094536708 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb0a0d640 1 --2- 172.21.15.83:0/2094536708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9eac101ff0 0x7f9eac1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb1a0f640 1 --2- 172.21.15.83:0/2094536708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eac1029b0 0x7f9eac19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb120e640 1 -- 172.21.15.83:0/2094536708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9eac101ff0 msgr2=0x7f9eac1975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb120e640 1 --2- 172.21.15.83:0/2094536708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9eac101ff0 0x7f9eac1975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:26.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb120e640 1 -- 172.21.15.83:0/2094536708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eac1029b0 msgr2=0x7f9eac19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:26.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb120e640 1 --2- 172.21.15.83:0/2094536708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eac1029b0 0x7f9eac19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:26.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb120e640 1 -- 172.21.15.83:0/2094536708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac19ec80 con 0x7f9eac100ea0 2024-04-25T11:41:26.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb1a0f640 1 --2- 172.21.15.83:0/2094536708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eac1029b0 0x7f9eac19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:26.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb120e640 1 --2- 172.21.15.83:0/2094536708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eac100ea0 0x7f9eac197060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00e9f0 tx=0x7f9e9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:26.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9ea27fc640 1 -- 172.21.15.83:0/2094536708 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00cd80 con 0x7f9eac100ea0 2024-04-25T11:41:26.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9ea27fc640 1 -- 172.21.15.83:0/2094536708 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e9c0108e0 con 0x7f9eac100ea0 2024-04-25T11:41:26.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9ea27fc640 1 -- 172.21.15.83:0/2094536708 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c018980 con 0x7f9eac100ea0 2024-04-25T11:41:26.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb3472640 1 -- 172.21.15.83:0/2094536708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac19ef70 con 0x7f9eac100ea0 2024-04-25T11:41:26.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.774+0000 7f9eb3472640 1 -- 172.21.15.83:0/2094536708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac19f430 con 0x7f9eac100ea0 2024-04-25T11:41:26.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.775+0000 7f9eb3472640 1 -- 172.21.15.83:0/2094536708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eac1081b0 con 0x7f9eac100ea0 2024-04-25T11:41:26.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.777+0000 7f9ea27fc640 1 -- 172.21.15.83:0/2094536708 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9e9c010430 con 0x7f9eac100ea0 2024-04-25T11:41:26.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.777+0000 7f9ea27fc640 1 --2- 172.21.15.83:0/2094536708 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9e980a3c30 0x7f9e980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:26.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.777+0000 7f9eb0a0d640 1 --2- 172.21.15.83:0/2094536708 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9e980a3c30 0x7f9e980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:26.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.777+0000 7f9ea27fc640 1 -- 172.21.15.83:0/2094536708 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9e9c0d0750 con 0x7f9eac100ea0 2024-04-25T11:41:26.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.778+0000 7f9eb0a0d640 1 --2- 172.21.15.83:0/2094536708 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9e980a3c30 0x7f9e980a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9eac198570 tx=0x7f9ea800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:26.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.781+0000 7f9ea27fc640 1 -- 172.21.15.83:0/2094536708 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e9c09d5d0 con 0x7f9eac100ea0 2024-04-25T11:41:26.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:26 smithi111 ceph-mon[32784]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:26.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:26 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:26.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:26 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:26.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:26 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:26.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:26 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:26.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:26 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:26.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:26 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:26.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:26 smithi083 ceph-mon[29330]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:26.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:26 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:26.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:26 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:26.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:26 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:26.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:26 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:26.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:26 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:26.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:26 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:26.950 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:26 smithi094 ceph-mon[33032]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:26.950 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:26 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:26.950 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:26 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:26.950 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:26 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:26.950 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:26 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:26.950 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:26 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:26.950 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:26 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:26.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.999+0000 7f9eb3472640 1 -- 172.21.15.83:0/2094536708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9eac02d5c0 con 0x7f9eac100ea0 2024-04-25T11:41:26.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:26.999+0000 7f9ea27fc640 1 -- 172.21.15.83:0/2094536708 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f9e9c09ced0 con 0x7f9eac100ea0 2024-04-25T11:41:26.986 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:26.986 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2927430878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2927430878},{"type":"v1","addr":"172.21.15.111:6837","nonce":2927430878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T11:41:26.986 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2024-04-25T11:41:26.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:27.001+0000 7f9eb3472640 1 -- 172.21.15.83:0/2094536708 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9e980a3c30 msgr2=0x7f9e980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:26.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:27.001+0000 7f9eb3472640 1 --2- 172.21.15.83:0/2094536708 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9e980a3c30 0x7f9e980a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9eac198570 tx=0x7f9ea800b410 comp rx=0 tx=0).stop 2024-04-25T11:41:26.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:27.001+0000 7f9eb3472640 1 -- 172.21.15.83:0/2094536708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eac100ea0 msgr2=0x7f9eac197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:26.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:27.001+0000 7f9eb3472640 1 --2- 172.21.15.83:0/2094536708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eac100ea0 0x7f9eac197060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00e9f0 tx=0x7f9e9c00eeb0 comp rx=0 tx=0).stop 2024-04-25T11:41:26.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:27.001+0000 7f9eb3472640 1 -- 172.21.15.83:0/2094536708 shutdown_connections 2024-04-25T11:41:26.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:27.001+0000 7f9eb3472640 1 --2- 172.21.15.83:0/2094536708 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9e980a3c30 0x7f9e980a60f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:26.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:27.001+0000 7f9eb3472640 1 --2- 172.21.15.83:0/2094536708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9eac1029b0 0x7f9eac19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:26.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:27.001+0000 7f9eb3472640 1 --2- 172.21.15.83:0/2094536708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9eac101ff0 0x7f9eac1975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:26.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:27.001+0000 7f9eb3472640 1 --2- 172.21.15.83:0/2094536708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eac100ea0 0x7f9eac197060 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:26.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:27.001+0000 7f9eb3472640 1 -- 172.21.15.83:0/2094536708 >> 172.21.15.83:0/2094536708 conn(0x7f9eac0fc430 msgr2=0x7f9eac105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:26.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:27.001+0000 7f9eb3472640 1 -- 172.21.15.83:0/2094536708 shutdown_connections 2024-04-25T11:41:26.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:27.001+0000 7f9eb3472640 1 -- 172.21.15.83:0/2094536708 wait complete. 2024-04-25T11:41:27.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:27 smithi111 ceph-mon[32784]: Deploying daemon mds.h on smithi094 2024-04-25T11:41:27.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:27 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] up:boot 2024-04-25T11:41:27.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:27 smithi111 ceph-mon[32784]: fsmap 7 up:standby 2024-04-25T11:41:27.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:27 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T11:41:27.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:27 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2094536708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:27.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:27 smithi083 ceph-mon[29330]: Deploying daemon mds.h on smithi094 2024-04-25T11:41:27.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:27 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] up:boot 2024-04-25T11:41:27.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:27 smithi083 ceph-mon[29330]: fsmap 7 up:standby 2024-04-25T11:41:27.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:27 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T11:41:27.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:27 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2094536708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:27.960 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:27 smithi094 ceph-mon[33032]: Deploying daemon mds.h on smithi094 2024-04-25T11:41:27.960 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:27 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] up:boot 2024-04-25T11:41:27.960 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:27 smithi094 ceph-mon[33032]: fsmap 7 up:standby 2024-04-25T11:41:27.960 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:27 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T11:41:27.961 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:27 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2094536708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:28.523 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:41:28 smithi094 systemd[1]: Starting Ceph mds.h for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:41:28.804 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:28 smithi094 ceph-mon[33032]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:28.804 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:41:28 smithi094 podman[62946]: 2024-04-25 11:41:28.573484115 +0000 UTC m=+0.019895962 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:41:28.804 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:41:28 smithi094 podman[62946]: 2024-04-25 11:41:28.679701897 +0000 UTC m=+0.126113726 container create 0ad77a914ff4be827de2e7762841217e343fdb7340c5203859661724fbe0d514 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T11:41:28.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:28 smithi111 ceph-mon[32784]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:28.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:28 smithi083 ceph-mon[29330]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:29.000 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:29.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.125+0000 7f62a8dad640 1 -- 172.21.15.83:0/843114284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a4100f10 msgr2=0x7f62a4101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:29.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.125+0000 7f62a8dad640 1 --2- 172.21.15.83:0/843114284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a4100f10 0x7f62a4101330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6294009900 tx=0x7f629402a410 comp rx=0 tx=0).stop 2024-04-25T11:41:29.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.125+0000 7f62a8dad640 1 -- 172.21.15.83:0/843114284 shutdown_connections 2024-04-25T11:41:29.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.125+0000 7f62a8dad640 1 --2- 172.21.15.83:0/843114284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4102a20 0x7f62a4109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:29.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.125+0000 7f62a8dad640 1 --2- 172.21.15.83:0/843114284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62a4102060 0x7f62a41024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:29.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.125+0000 7f62a8dad640 1 --2- 172.21.15.83:0/843114284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a4100f10 0x7f62a4101330 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:29.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.125+0000 7f62a8dad640 1 -- 172.21.15.83:0/843114284 >> 172.21.15.83:0/843114284 conn(0x7f62a40fc4a0 msgr2=0x7f62a40fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:29.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.125+0000 7f62a8dad640 1 -- 172.21.15.83:0/843114284 shutdown_connections 2024-04-25T11:41:29.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.125+0000 7f62a8dad640 1 -- 172.21.15.83:0/843114284 wait complete. 2024-04-25T11:41:29.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a8dad640 1 Processor -- start 2024-04-25T11:41:29.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a8dad640 1 -- start start 2024-04-25T11:41:29.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a8dad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a4100f10 0x7f62a410b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:29.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a8dad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4102060 0x7f62a4109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:29.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a8dad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62a4102a20 0x7f62a410a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:29.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a8dad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62a410db70 con 0x7f62a4100f10 2024-04-25T11:41:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a8dad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f62a410d9f0 con 0x7f62a4102a20 2024-04-25T11:41:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a8dad640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f62a410dcf0 con 0x7f62a4102060 2024-04-25T11:41:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a4100f10 0x7f62a410b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a4100f10 0x7f62a410b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46860/0 (socket says 172.21.15.83:46860) 2024-04-25T11:41:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a259c640 1 -- 172.21.15.83:0/3501644409 learned_addr learned my addr 172.21.15.83:0/3501644409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a2d9d640 1 --2- 172.21.15.83:0/3501644409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62a4102a20 0x7f62a410a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f6299d9b640 1 --2- 172.21.15.83:0/3501644409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4102060 0x7f62a4109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a259c640 1 -- 172.21.15.83:0/3501644409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4102060 msgr2=0x7f62a4109ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a259c640 1 --2- 172.21.15.83:0/3501644409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4102060 0x7f62a4109ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a259c640 1 -- 172.21.15.83:0/3501644409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62a4102a20 msgr2=0x7f62a410a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a259c640 1 --2- 172.21.15.83:0/3501644409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62a4102a20 0x7f62a410a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.126+0000 7f62a259c640 1 -- 172.21.15.83:0/3501644409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a410acc0 con 0x7f62a4100f10 2024-04-25T11:41:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.127+0000 7f62a259c640 1 --2- 172.21.15.83:0/3501644409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a4100f10 0x7f62a410b8b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6294005940 tx=0x7f6294004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.127+0000 7f629bfff640 1 -- 172.21.15.83:0/3501644409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629403f030 con 0x7f62a4100f10 2024-04-25T11:41:29.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.127+0000 7f629bfff640 1 -- 172.21.15.83:0/3501644409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6294003cb0 con 0x7f62a4100f10 2024-04-25T11:41:29.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.127+0000 7f62a8dad640 1 -- 172.21.15.83:0/3501644409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a41a2360 con 0x7f62a4100f10 2024-04-25T11:41:29.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.127+0000 7f629bfff640 1 -- 172.21.15.83:0/3501644409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294035070 con 0x7f62a4100f10 2024-04-25T11:41:29.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.127+0000 7f62a8dad640 1 -- 172.21.15.83:0/3501644409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a41a28b0 con 0x7f62a4100f10 2024-04-25T11:41:29.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.128+0000 7f62a8dad640 1 -- 172.21.15.83:0/3501644409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a41082f0 con 0x7f62a4100f10 2024-04-25T11:41:29.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.129+0000 7f629bfff640 1 -- 172.21.15.83:0/3501644409 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6294042030 con 0x7f62a4100f10 2024-04-25T11:41:29.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.129+0000 7f629bfff640 1 --2- 172.21.15.83:0/3501644409 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f62900a3bb0 0x7f62900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:29.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.130+0000 7f6299d9b640 1 --2- 172.21.15.83:0/3501644409 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f62900a3bb0 0x7f62900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:29.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.130+0000 7f629bfff640 1 -- 172.21.15.83:0/3501644409 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f62940f0460 con 0x7f62a4100f10 2024-04-25T11:41:29.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.130+0000 7f6299d9b640 1 --2- 172.21.15.83:0/3501644409 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f62900a3bb0 0x7f62900a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f62a410b620 tx=0x7f628c009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:29.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.134+0000 7f629bfff640 1 -- 172.21.15.83:0/3501644409 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62940bd2e0 con 0x7f62a4100f10 2024-04-25T11:41:29.272 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:41:28 smithi094 podman[62946]: 2024-04-25 11:41:28.954910856 +0000 UTC m=+0.401322685 container init 0ad77a914ff4be827de2e7762841217e343fdb7340c5203859661724fbe0d514 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T11:41:29.273 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:41:28 smithi094 podman[62946]: 2024-04-25 11:41:28.957521996 +0000 UTC m=+0.403933831 container start 0ad77a914ff4be827de2e7762841217e343fdb7340c5203859661724fbe0d514 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T11:41:29.273 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:41:28 smithi094 ceph-mds[62965]: -- 172.21.15.94:0/2038049748 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e360e1da40 con 0x55e360df1000 2024-04-25T11:41:29.273 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:41:28 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-h[62961]: starting mds.h at 2024-04-25T11:41:29.273 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:41:29 smithi094 bash[62946]: 0ad77a914ff4be827de2e7762841217e343fdb7340c5203859661724fbe0d514 2024-04-25T11:41:29.273 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:41:29 smithi094 systemd[1]: Started Ceph mds.h for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:41:29.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.351+0000 7f62a8dad640 1 -- 172.21.15.83:0/3501644409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62a402d7c0 con 0x7f62a4100f10 2024-04-25T11:41:29.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.351+0000 7f629bfff640 1 -- 172.21.15.83:0/3501644409 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f62940bcbe0 con 0x7f62a4100f10 2024-04-25T11:41:29.338 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:29.338 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2927430878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2927430878},{"type":"v1","addr":"172.21.15.111:6837","nonce":2927430878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T11:41:29.339 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2024-04-25T11:41:29.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.353+0000 7f62a8dad640 1 -- 172.21.15.83:0/3501644409 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f62900a3bb0 msgr2=0x7f62900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:29.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.354+0000 7f62a8dad640 1 --2- 172.21.15.83:0/3501644409 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f62900a3bb0 0x7f62900a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f62a410b620 tx=0x7f628c009110 comp rx=0 tx=0).stop 2024-04-25T11:41:29.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.354+0000 7f62a8dad640 1 -- 172.21.15.83:0/3501644409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a4100f10 msgr2=0x7f62a410b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:29.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.354+0000 7f62a8dad640 1 --2- 172.21.15.83:0/3501644409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a4100f10 0x7f62a410b8b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6294005940 tx=0x7f6294004430 comp rx=0 tx=0).stop 2024-04-25T11:41:29.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.354+0000 7f62a8dad640 1 -- 172.21.15.83:0/3501644409 shutdown_connections 2024-04-25T11:41:29.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.354+0000 7f62a8dad640 1 --2- 172.21.15.83:0/3501644409 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f62900a3bb0 0x7f62900a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:29.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.354+0000 7f62a8dad640 1 --2- 172.21.15.83:0/3501644409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f62a4102a20 0x7f62a410a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:29.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.354+0000 7f62a8dad640 1 --2- 172.21.15.83:0/3501644409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62a4102060 0x7f62a4109ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:29.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.354+0000 7f62a8dad640 1 --2- 172.21.15.83:0/3501644409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62a4100f10 0x7f62a410b8b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:29.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.354+0000 7f62a8dad640 1 -- 172.21.15.83:0/3501644409 >> 172.21.15.83:0/3501644409 conn(0x7f62a40fc4a0 msgr2=0x7f62a41054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:29.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.354+0000 7f62a8dad640 1 -- 172.21.15.83:0/3501644409 shutdown_connections 2024-04-25T11:41:29.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:29.354+0000 7f62a8dad640 1 -- 172.21.15.83:0/3501644409 wait complete. 2024-04-25T11:41:30.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:30 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:30.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:30 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:30 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:30 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:30 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:30 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:30 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3501644409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:30 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] up:boot 2024-04-25T11:41:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:30 smithi111 ceph-mon[32784]: fsmap 8 up:standby 2024-04-25T11:41:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:30 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T11:41:30.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:30.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:30.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:30.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:30.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:30.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:30.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:30 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3501644409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:30.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:30 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] up:boot 2024-04-25T11:41:30.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:30 smithi083 ceph-mon[29330]: fsmap 8 up:standby 2024-04-25T11:41:30.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:30 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T11:41:30.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:30 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:30.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:30 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:30.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:30 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:30.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:30 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:30.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:30 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:30.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:30 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:30.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:30 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3501644409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:30.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:30 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] up:boot 2024-04-25T11:41:30.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:30 smithi094 ceph-mon[33032]: fsmap 8 up:standby 2024-04-25T11:41:30.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:30 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T11:41:31.284 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:31 smithi111 ceph-mon[32784]: Deploying daemon mds.i on smithi111 2024-04-25T11:41:31.284 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:31 smithi111 ceph-mon[32784]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:31.352 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:31.377 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:31 smithi083 ceph-mon[29330]: Deploying daemon mds.i on smithi111 2024-04-25T11:41:31.377 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:31 smithi083 ceph-mon[29330]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:31.424 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:31 smithi094 ceph-mon[33032]: Deploying daemon mds.i on smithi111 2024-04-25T11:41:31.424 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:31 smithi094 ceph-mon[33032]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:31.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.476+0000 7f047a0cb640 1 -- 172.21.15.83:0/3707408742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474100e90 msgr2=0x7f04741012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:31.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.476+0000 7f047a0cb640 1 --2- 172.21.15.83:0/3707408742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474100e90 0x7f04741012b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0464009900 tx=0x7f046402a410 comp rx=0 tx=0).stop 2024-04-25T11:41:31.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.476+0000 7f047a0cb640 1 -- 172.21.15.83:0/3707408742 shutdown_connections 2024-04-25T11:41:31.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.476+0000 7f047a0cb640 1 --2- 172.21.15.83:0/3707408742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04741029a0 0x7f04741092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:31.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.476+0000 7f047a0cb640 1 --2- 172.21.15.83:0/3707408742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0474101fe0 0x7f0474102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:31.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.476+0000 7f047a0cb640 1 --2- 172.21.15.83:0/3707408742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474100e90 0x7f04741012b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.476+0000 7f047a0cb640 1 -- 172.21.15.83:0/3707408742 >> 172.21.15.83:0/3707408742 conn(0x7f04740fc400 msgr2=0x7f04740fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.476+0000 7f047a0cb640 1 -- 172.21.15.83:0/3707408742 shutdown_connections 2024-04-25T11:41:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.476+0000 7f047a0cb640 1 -- 172.21.15.83:0/3707408742 wait complete. 2024-04-25T11:41:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.476+0000 7f047a0cb640 1 Processor -- start 2024-04-25T11:41:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.476+0000 7f047a0cb640 1 -- start start 2024-04-25T11:41:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.476+0000 7f047a0cb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0474100e90 0x7f0474069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.476+0000 7f047a0cb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474101fe0 0x7f047419cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.476+0000 7f047a0cb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04741029a0 0x7f047419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f047a0cb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f047410c9e0 con 0x7f0474101fe0 2024-04-25T11:41:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f047a0cb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f047410c860 con 0x7f04741029a0 2024-04-25T11:41:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f047a0cb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f047410cb60 con 0x7f0474100e90 2024-04-25T11:41:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f046bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474101fe0 0x7f047419cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f046bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474101fe0 0x7f047419cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46878/0 (socket says 172.21.15.83:46878) 2024-04-25T11:41:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f046bfff640 1 -- 172.21.15.83:0/1792641863 learned_addr learned my addr 172.21.15.83:0/1792641863 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f04737fe640 1 --2- 172.21.15.83:0/1792641863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0474100e90 0x7f0474069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f0473fff640 1 --2- 172.21.15.83:0/1792641863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04741029a0 0x7f047419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f046bfff640 1 -- 172.21.15.83:0/1792641863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0474100e90 msgr2=0x7f0474069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f046bfff640 1 --2- 172.21.15.83:0/1792641863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0474100e90 0x7f0474069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f046bfff640 1 -- 172.21.15.83:0/1792641863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04741029a0 msgr2=0x7f047419f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f046bfff640 1 --2- 172.21.15.83:0/1792641863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04741029a0 0x7f047419f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f046bfff640 1 -- 172.21.15.83:0/1792641863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047419fe80 con 0x7f0474101fe0 2024-04-25T11:41:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f0473fff640 1 --2- 172.21.15.83:0/1792641863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04741029a0 0x7f047419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f046bfff640 1 --2- 172.21.15.83:0/1792641863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474101fe0 0x7f047419cf50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f045c00e9c0 tx=0x7f045c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f04717fa640 1 -- 172.21.15.83:0/1792641863 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c00cd80 con 0x7f0474101fe0 2024-04-25T11:41:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f04717fa640 1 -- 172.21.15.83:0/1792641863 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f045c0035f0 con 0x7f0474101fe0 2024-04-25T11:41:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f047a0cb640 1 -- 172.21.15.83:0/1792641863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04741a0110 con 0x7f0474101fe0 2024-04-25T11:41:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f04717fa640 1 -- 172.21.15.83:0/1792641863 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c010690 con 0x7f0474101fe0 2024-04-25T11:41:31.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.477+0000 7f047a0cb640 1 -- 172.21.15.83:0/1792641863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04741a0610 con 0x7f0474101fe0 2024-04-25T11:41:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.478+0000 7f047a0cb640 1 -- 172.21.15.83:0/1792641863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0474108270 con 0x7f0474101fe0 2024-04-25T11:41:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.479+0000 7f04717fa640 1 -- 172.21.15.83:0/1792641863 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f045c004000 con 0x7f0474101fe0 2024-04-25T11:41:31.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.480+0000 7f04717fa640 1 --2- 172.21.15.83:0/1792641863 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f04600a3bb0 0x7f04600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:31.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.480+0000 7f04737fe640 1 --2- 172.21.15.83:0/1792641863 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f04600a3bb0 0x7f04600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:31.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.480+0000 7f04717fa640 1 -- 172.21.15.83:0/1792641863 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f045c0d0680 con 0x7f0474101fe0 2024-04-25T11:41:31.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.480+0000 7f04737fe640 1 --2- 172.21.15.83:0/1792641863 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f04600a3bb0 0x7f04600a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0464000c00 tx=0x7f04640314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:31.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.483+0000 7f04717fa640 1 -- 172.21.15.83:0/1792641863 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f045c09d500 con 0x7f0474101fe0 2024-04-25T11:41:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.700+0000 7f047a0cb640 1 -- 172.21.15.83:0/1792641863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f047402d5c0 con 0x7f0474101fe0 2024-04-25T11:41:31.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.701+0000 7f04717fa640 1 -- 172.21.15.83:0/1792641863 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7177 (secure 0 0 0) 0x7f045c09ce00 con 0x7f0474101fe0 2024-04-25T11:41:31.688 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:31.688 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2475976592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2475976592},{"type":"v1","addr":"172.21.15.94:6837","nonce":2475976592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2927430878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2927430878},{"type":"v1","addr":"172.21.15.111:6837","nonce":2927430878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T11:41:31.688 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2024-04-25T11:41:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.703+0000 7f047a0cb640 1 -- 172.21.15.83:0/1792641863 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f04600a3bb0 msgr2=0x7f04600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.703+0000 7f047a0cb640 1 --2- 172.21.15.83:0/1792641863 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f04600a3bb0 0x7f04600a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0464000c00 tx=0x7f04640314a0 comp rx=0 tx=0).stop 2024-04-25T11:41:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.703+0000 7f047a0cb640 1 -- 172.21.15.83:0/1792641863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474101fe0 msgr2=0x7f047419cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.703+0000 7f047a0cb640 1 --2- 172.21.15.83:0/1792641863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474101fe0 0x7f047419cf50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f045c00e9c0 tx=0x7f045c00ee80 comp rx=0 tx=0).stop 2024-04-25T11:41:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.703+0000 7f047a0cb640 1 -- 172.21.15.83:0/1792641863 shutdown_connections 2024-04-25T11:41:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.703+0000 7f047a0cb640 1 --2- 172.21.15.83:0/1792641863 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f04600a3bb0 0x7f04600a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.703+0000 7f047a0cb640 1 --2- 172.21.15.83:0/1792641863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0474101fe0 0x7f047419cf50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.703+0000 7f047a0cb640 1 --2- 172.21.15.83:0/1792641863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f04741029a0 0x7f047419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.703+0000 7f047a0cb640 1 --2- 172.21.15.83:0/1792641863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0474100e90 0x7f0474069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.703+0000 7f047a0cb640 1 -- 172.21.15.83:0/1792641863 >> 172.21.15.83:0/1792641863 conn(0x7f04740fc400 msgr2=0x7f0474105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.703+0000 7f047a0cb640 1 -- 172.21.15.83:0/1792641863 shutdown_connections 2024-04-25T11:41:31.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:31.703+0000 7f047a0cb640 1 -- 172.21.15.83:0/1792641863 wait complete. 2024-04-25T11:41:32.068 INFO:journalctl@ceph.mds.i.smithi111.stdout:Apr 25 11:41:31 smithi111 systemd[1]: Started Ceph mds.i for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:41:32.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:32 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1792641863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:32.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:32 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:32.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:32 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:32.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:32 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:32.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:32 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:32.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:32 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:32.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:32 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:32.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:32 smithi111 ceph-mon[32784]: Deploying daemon mds.g on smithi083 2024-04-25T11:41:32.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:32 smithi111 ceph-mon[32784]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:32.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:32 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1792641863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:32.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:32 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:32.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:32 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:32.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:32 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:32.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:32 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:32.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:32 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:32.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:32 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:32.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:32 smithi083 ceph-mon[29330]: Deploying daemon mds.g on smithi083 2024-04-25T11:41:32.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:32 smithi083 ceph-mon[29330]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:32.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:32 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1792641863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:32.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:32 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:32.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:32 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:32.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:32 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:32.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:32 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:32.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:32 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:32.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:32 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:32.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:32 smithi094 ceph-mon[33032]: Deploying daemon mds.g on smithi083 2024-04-25T11:41:32.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:32 smithi094 ceph-mon[33032]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:33.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:33 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] up:boot 2024-04-25T11:41:33.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:33 smithi083 ceph-mon[29330]: fsmap 9 up:standby 2024-04-25T11:41:33.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:33 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:41:33.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:33 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] up:boot 2024-04-25T11:41:33.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:33 smithi094 ceph-mon[33032]: fsmap 9 up:standby 2024-04-25T11:41:33.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:33 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:41:33.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:33 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] up:boot 2024-04-25T11:41:33.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:33 smithi111 ceph-mon[32784]: fsmap 9 up:standby 2024-04-25T11:41:33.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:33 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:41:33.702 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.916+0000 7f1e5d4a7640 1 -- 172.21.15.83:0/2758095071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e58107d50 msgr2=0x7f1e5810a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.916+0000 7f1e5d4a7640 1 --2- 172.21.15.83:0/2758095071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e58107d50 0x7f1e5810a140 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009ca0 tx=0x7f1e5002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.917+0000 7f1e5d4a7640 1 -- 172.21.15.83:0/2758095071 shutdown_connections 2024-04-25T11:41:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.917+0000 7f1e5d4a7640 1 --2- 172.21.15.83:0/2758095071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e58107d50 0x7f1e5810a140 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.917+0000 7f1e5d4a7640 1 --2- 172.21.15.83:0/2758095071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e58105420 0x7f1e58107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.917+0000 7f1e5d4a7640 1 --2- 172.21.15.83:0/2758095071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58102af0 0x7f1e58104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.917+0000 7f1e5d4a7640 1 -- 172.21.15.83:0/2758095071 >> 172.21.15.83:0/2758095071 conn(0x7f1e580fc440 msgr2=0x7f1e580fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.917+0000 7f1e5d4a7640 1 -- 172.21.15.83:0/2758095071 shutdown_connections 2024-04-25T11:41:33.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.917+0000 7f1e5d4a7640 1 -- 172.21.15.83:0/2758095071 wait complete. 2024-04-25T11:41:33.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.917+0000 7f1e5d4a7640 1 Processor -- start 2024-04-25T11:41:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e5d4a7640 1 -- start start 2024-04-25T11:41:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e5d4a7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58102af0 0x7f1e581981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e5d4a7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e58105420 0x7f1e581986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e5d4a7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e58107d50 0x7f1e5819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e5d4a7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5810db00 con 0x7f1e58105420 2024-04-25T11:41:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e5d4a7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5810d980 con 0x7f1e58102af0 2024-04-25T11:41:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e5d4a7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5810dc80 con 0x7f1e58107d50 2024-04-25T11:41:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e567fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e58105420 0x7f1e581986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e567fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e58105420 0x7f1e581986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43874/0 (socket says 172.21.15.83:43874) 2024-04-25T11:41:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e567fc640 1 -- 172.21.15.83:0/405903482 learned_addr learned my addr 172.21.15.83:0/405903482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e577fe640 1 --2- 172.21.15.83:0/405903482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e58107d50 0x7f1e5819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e567fc640 1 -- 172.21.15.83:0/405903482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e58107d50 msgr2=0x7f1e5819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e56ffd640 1 --2- 172.21.15.83:0/405903482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58102af0 0x7f1e581981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e567fc640 1 --2- 172.21.15.83:0/405903482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e58107d50 0x7f1e5819f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e567fc640 1 -- 172.21.15.83:0/405903482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58102af0 msgr2=0x7f1e581981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e567fc640 1 --2- 172.21.15.83:0/405903482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58102af0 0x7f1e581981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e567fc640 1 -- 172.21.15.83:0/405903482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5819fdd0 con 0x7f1e58105420 2024-04-25T11:41:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e567fc640 1 --2- 172.21.15.83:0/405903482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e58105420 0x7f1e581986f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800d740 tx=0x7f1e4800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e3ffff640 1 -- 172.21.15.83:0/405903482 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48017070 con 0x7f1e58105420 2024-04-25T11:41:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e5d4a7640 1 -- 172.21.15.83:0/405903482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e581a00c0 con 0x7f1e58105420 2024-04-25T11:41:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e3ffff640 1 -- 172.21.15.83:0/405903482 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4800de50 con 0x7f1e58105420 2024-04-25T11:41:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.918+0000 7f1e3ffff640 1 -- 172.21.15.83:0/405903482 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48004ce0 con 0x7f1e58105420 2024-04-25T11:41:33.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.919+0000 7f1e5d4a7640 1 -- 172.21.15.83:0/405903482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e581a0610 con 0x7f1e58105420 2024-04-25T11:41:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.919+0000 7f1e5d4a7640 1 -- 172.21.15.83:0/405903482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e580fdf30 con 0x7f1e58105420 2024-04-25T11:41:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.920+0000 7f1e3ffff640 1 -- 172.21.15.83:0/405903482 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1e48004e80 con 0x7f1e58105420 2024-04-25T11:41:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.921+0000 7f1e3ffff640 1 --2- 172.21.15.83:0/405903482 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1e400a3bb0 0x7f1e400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.921+0000 7f1e56ffd640 1 --2- 172.21.15.83:0/405903482 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1e400a3bb0 0x7f1e400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:33.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.921+0000 7f1e3ffff640 1 -- 172.21.15.83:0/405903482 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1e480cfea0 con 0x7f1e58105420 2024-04-25T11:41:33.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.922+0000 7f1e56ffd640 1 --2- 172.21.15.83:0/405903482 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1e400a3bb0 0x7f1e400a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0058c0 tx=0x7f1e4c005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:33.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:33.925+0000 7f1e3ffff640 1 -- 172.21.15.83:0/405903482 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e4809cc80 con 0x7f1e58105420 2024-04-25T11:41:34.024 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:41:34 smithi083 systemd[1]: Starting Ceph mds.g for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:41:34.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:34.156+0000 7f1e5d4a7640 1 -- 172.21.15.83:0/405903482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e5802d5c0 con 0x7f1e58105420 2024-04-25T11:41:34.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:34.157+0000 7f1e3ffff640 1 -- 172.21.15.83:0/405903482 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7f1e4809c580 con 0x7f1e58105420 2024-04-25T11:41:34.144 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:34.144 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2475976592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2475976592},{"type":"v1","addr":"172.21.15.94:6837","nonce":2475976592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2927430878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2927430878},{"type":"v1","addr":"172.21.15.111:6837","nonce":2927430878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/2233531151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2233531151},{"type":"v1","addr":"172.21.15.111:6839","nonce":2233531151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T11:41:34.144 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 10 2024-04-25T11:41:34.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:34.159+0000 7f1e5d4a7640 1 -- 172.21.15.83:0/405903482 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1e400a3bb0 msgr2=0x7f1e400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:34.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:34.159+0000 7f1e5d4a7640 1 --2- 172.21.15.83:0/405903482 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1e400a3bb0 0x7f1e400a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0058c0 tx=0x7f1e4c005c60 comp rx=0 tx=0).stop 2024-04-25T11:41:34.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:34.159+0000 7f1e5d4a7640 1 -- 172.21.15.83:0/405903482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e58105420 msgr2=0x7f1e581986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:34.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:34.159+0000 7f1e5d4a7640 1 --2- 172.21.15.83:0/405903482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e58105420 0x7f1e581986f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800d740 tx=0x7f1e4800dc00 comp rx=0 tx=0).stop 2024-04-25T11:41:34.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:34.159+0000 7f1e5d4a7640 1 -- 172.21.15.83:0/405903482 shutdown_connections 2024-04-25T11:41:34.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:34.159+0000 7f1e5d4a7640 1 --2- 172.21.15.83:0/405903482 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1e400a3bb0 0x7f1e400a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:34.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:34.159+0000 7f1e5d4a7640 1 --2- 172.21.15.83:0/405903482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e58105420 0x7f1e581986f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:34.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:34.159+0000 7f1e5d4a7640 1 --2- 172.21.15.83:0/405903482 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e58107d50 0x7f1e5819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:34.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:34.159+0000 7f1e5d4a7640 1 --2- 172.21.15.83:0/405903482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e58102af0 0x7f1e581981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:34.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:34.159+0000 7f1e5d4a7640 1 -- 172.21.15.83:0/405903482 >> 172.21.15.83:0/405903482 conn(0x7f1e580fc440 msgr2=0x7f1e58103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:34.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:34.159+0000 7f1e5d4a7640 1 -- 172.21.15.83:0/405903482 shutdown_connections 2024-04-25T11:41:34.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:34.159+0000 7f1e5d4a7640 1 -- 172.21.15.83:0/405903482 wait complete. 2024-04-25T11:41:34.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:34 smithi083 ceph-mon[29330]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:34.386 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:41:34 smithi083 podman[71360]: 2024-04-25 11:41:34.161107211 +0000 UTC m=+0.019030516 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:41:34.386 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:41:34 smithi083 podman[71360]: 2024-04-25 11:41:34.272028702 +0000 UTC m=+0.129952005 container create 1a1f96ec98c9050cc5365245aa96e09c5b071dd467517483841ea0dc8e1d94ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T11:41:34.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:34 smithi094 ceph-mon[33032]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:34.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:34 smithi111 ceph-mon[32784]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:34.655 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:41:34 smithi083 podman[71360]: 2024-04-25 11:41:34.508345664 +0000 UTC m=+0.366268967 container init 1a1f96ec98c9050cc5365245aa96e09c5b071dd467517483841ea0dc8e1d94ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T11:41:34.655 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:41:34 smithi083 podman[71360]: 2024-04-25 11:41:34.510814744 +0000 UTC m=+0.368738057 container start 1a1f96ec98c9050cc5365245aa96e09c5b071dd467517483841ea0dc8e1d94ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:41:34.655 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:41:34 smithi083 ceph-mds[71382]: -- 172.21.15.83:0/901536938 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d8cc5b7a40 con 0x55d8cc58b000 2024-04-25T11:41:34.655 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:41:34 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-g[71378]: starting mds.g at 2024-04-25T11:41:34.656 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:41:34 smithi083 bash[71360]: 1a1f96ec98c9050cc5365245aa96e09c5b071dd467517483841ea0dc8e1d94ac 2024-04-25T11:41:34.656 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:41:34 smithi083 systemd[1]: Started Ceph mds.g for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:41:35.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:35 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/405903482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:35.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:35 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:35.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:35 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:35.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:35 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:35.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:35 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:35.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:35 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:35.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:35 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:35.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:35 smithi083 ceph-mon[29330]: Deploying daemon mds.a on smithi083 2024-04-25T11:41:35.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:35 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/405903482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:35.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:35 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:35.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:35 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:35.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:35 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:35.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:35 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:35.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:35 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:35.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:35 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:35.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:35 smithi094 ceph-mon[33032]: Deploying daemon mds.a on smithi083 2024-04-25T11:41:35.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:35 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/405903482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:35.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:35 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:35.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:35 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:35.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:35 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:35.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:35 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:35.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:35 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:35.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:35 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:35.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:35 smithi111 ceph-mon[32784]: Deploying daemon mds.a on smithi083 2024-04-25T11:41:36.159 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:36.236 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:36 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.83:6838/1568064835,v1:172.21.15.83:6839/1568064835] up:boot 2024-04-25T11:41:36.236 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:36 smithi083 ceph-mon[29330]: fsmap 10 up:standby 2024-04-25T11:41:36.236 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:36 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T11:41:36.236 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:36 smithi083 ceph-mon[29330]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:36.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.277+0000 7f7357515640 1 -- 172.21.15.83:0/1944669069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7350100e90 msgr2=0x7f73501012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.277+0000 7f7357515640 1 --2- 172.21.15.83:0/1944669069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7350100e90 0x7f73501012b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f734c009900 tx=0x7f734c02a410 comp rx=0 tx=0).stop 2024-04-25T11:41:36.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.277+0000 7f7357515640 1 -- 172.21.15.83:0/1944669069 shutdown_connections 2024-04-25T11:41:36.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.277+0000 7f7357515640 1 --2- 172.21.15.83:0/1944669069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73501029a0 0x7f73501092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.277+0000 7f7357515640 1 --2- 172.21.15.83:0/1944669069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7350101fe0 0x7f7350102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.277+0000 7f7357515640 1 --2- 172.21.15.83:0/1944669069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7350100e90 0x7f73501012b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.277+0000 7f7357515640 1 -- 172.21.15.83:0/1944669069 >> 172.21.15.83:0/1944669069 conn(0x7f73500fc400 msgr2=0x7f73500fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.277+0000 7f7357515640 1 -- 172.21.15.83:0/1944669069 shutdown_connections 2024-04-25T11:41:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.277+0000 7f7357515640 1 -- 172.21.15.83:0/1944669069 wait complete. 2024-04-25T11:41:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.277+0000 7f7357515640 1 Processor -- start 2024-04-25T11:41:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.277+0000 7f7357515640 1 -- start start 2024-04-25T11:41:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f7357515640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7350100e90 0x7f7350198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f7357515640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7350101fe0 0x7f7350198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f7357515640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73501029a0 0x7f735019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f7357515640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f735010c9e0 con 0x7f7350100e90 2024-04-25T11:41:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f7357515640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f735010c860 con 0x7f73501029a0 2024-04-25T11:41:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f7357515640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f735010cb60 con 0x7f7350101fe0 2024-04-25T11:41:36.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f73552b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7350100e90 0x7f7350198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f73552b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7350100e90 0x7f7350198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43900/0 (socket says 172.21.15.83:43900) 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f73552b1640 1 -- 172.21.15.83:0/2528971956 learned_addr learned my addr 172.21.15.83:0/2528971956 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f7354ab0640 1 --2- 172.21.15.83:0/2528971956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7350101fe0 0x7f7350198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f7355ab2640 1 --2- 172.21.15.83:0/2528971956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73501029a0 0x7f735019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f73552b1640 1 -- 172.21.15.83:0/2528971956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7350101fe0 msgr2=0x7f7350198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f73552b1640 1 --2- 172.21.15.83:0/2528971956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7350101fe0 0x7f7350198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f73552b1640 1 -- 172.21.15.83:0/2528971956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73501029a0 msgr2=0x7f735019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f73552b1640 1 --2- 172.21.15.83:0/2528971956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73501029a0 0x7f735019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f73552b1640 1 -- 172.21.15.83:0/2528971956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735019fe30 con 0x7f7350100e90 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f73552b1640 1 --2- 172.21.15.83:0/2528971956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7350100e90 0x7f7350198210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f734c005940 tx=0x7f734c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f73467fc640 1 -- 172.21.15.83:0/2528971956 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c03f030 con 0x7f7350100e90 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f73467fc640 1 -- 172.21.15.83:0/2528971956 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f734c003aa0 con 0x7f7350100e90 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f73467fc640 1 -- 172.21.15.83:0/2528971956 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734c035070 con 0x7f7350100e90 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f7357515640 1 -- 172.21.15.83:0/2528971956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73501a00c0 con 0x7f7350100e90 2024-04-25T11:41:36.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.278+0000 7f7357515640 1 -- 172.21.15.83:0/2528971956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73501a0600 con 0x7f7350100e90 2024-04-25T11:41:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.280+0000 7f7357515640 1 -- 172.21.15.83:0/2528971956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73501a0860 con 0x7f7350100e90 2024-04-25T11:41:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.280+0000 7f73467fc640 1 -- 172.21.15.83:0/2528971956 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f734c042030 con 0x7f7350100e90 2024-04-25T11:41:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.281+0000 7f73467fc640 1 --2- 172.21.15.83:0/2528971956 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f733c0a3c30 0x7f733c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.281+0000 7f7354ab0640 1 --2- 172.21.15.83:0/2528971956 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f733c0a3c30 0x7f733c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.281+0000 7f73467fc640 1 -- 172.21.15.83:0/2528971956 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f734c0f0520 con 0x7f7350100e90 2024-04-25T11:41:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.282+0000 7f7354ab0640 1 --2- 172.21.15.83:0/2528971956 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f733c0a3c30 0x7f733c0a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7350199720 tx=0x7f734000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:36.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.286+0000 7f73467fc640 1 -- 172.21.15.83:0/2528971956 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f734c0bd3a0 con 0x7f7350100e90 2024-04-25T11:41:36.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:36 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.83:6838/1568064835,v1:172.21.15.83:6839/1568064835] up:boot 2024-04-25T11:41:36.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:36 smithi094 ceph-mon[33032]: fsmap 10 up:standby 2024-04-25T11:41:36.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:36 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T11:41:36.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:36 smithi094 ceph-mon[33032]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:36.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.542+0000 7f7357515640 1 -- 172.21.15.83:0/2528971956 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73501a0a30 con 0x7f7350100e90 2024-04-25T11:41:36.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.543+0000 7f73467fc640 1 -- 172.21.15.83:0/2528971956 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7f734c0bcca0 con 0x7f7350100e90 2024-04-25T11:41:36.529 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:36.530 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1568064835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1568064835},{"type":"v1","addr":"172.21.15.83:6839","nonce":1568064835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2475976592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2475976592},{"type":"v1","addr":"172.21.15.94:6837","nonce":2475976592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2927430878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2927430878},{"type":"v1","addr":"172.21.15.111:6837","nonce":2927430878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/2233531151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2233531151},{"type":"v1","addr":"172.21.15.111:6839","nonce":2233531151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T11:41:36.530 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 11 2024-04-25T11:41:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.545+0000 7f7357515640 1 -- 172.21.15.83:0/2528971956 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f733c0a3c30 msgr2=0x7f733c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.545+0000 7f7357515640 1 --2- 172.21.15.83:0/2528971956 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f733c0a3c30 0x7f733c0a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7350199720 tx=0x7f734000b390 comp rx=0 tx=0).stop 2024-04-25T11:41:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.545+0000 7f7357515640 1 -- 172.21.15.83:0/2528971956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7350100e90 msgr2=0x7f7350198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.545+0000 7f7357515640 1 --2- 172.21.15.83:0/2528971956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7350100e90 0x7f7350198210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f734c005940 tx=0x7f734c031a50 comp rx=0 tx=0).stop 2024-04-25T11:41:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.545+0000 7f7357515640 1 -- 172.21.15.83:0/2528971956 shutdown_connections 2024-04-25T11:41:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.545+0000 7f7357515640 1 --2- 172.21.15.83:0/2528971956 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f733c0a3c30 0x7f733c0a60f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.545+0000 7f7357515640 1 --2- 172.21.15.83:0/2528971956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73501029a0 0x7f735019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.545+0000 7f7357515640 1 --2- 172.21.15.83:0/2528971956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7350101fe0 0x7f7350198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.545+0000 7f7357515640 1 --2- 172.21.15.83:0/2528971956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7350100e90 0x7f7350198210 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.545+0000 7f7357515640 1 -- 172.21.15.83:0/2528971956 >> 172.21.15.83:0/2528971956 conn(0x7f73500fc400 msgr2=0x7f73500fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.545+0000 7f7357515640 1 -- 172.21.15.83:0/2528971956 shutdown_connections 2024-04-25T11:41:36.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:36.545+0000 7f7357515640 1 -- 172.21.15.83:0/2528971956 wait complete. 2024-04-25T11:41:36.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:36 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.83:6838/1568064835,v1:172.21.15.83:6839/1568064835] up:boot 2024-04-25T11:41:36.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:36 smithi111 ceph-mon[32784]: fsmap 10 up:standby 2024-04-25T11:41:36.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:36 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T11:41:36.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:36 smithi111 ceph-mon[32784]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:36.905 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:41:36 smithi083 systemd[1]: Starting Ceph mds.a for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:41:37.190 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:37 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2528971956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:37.191 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:41:36 smithi083 podman[71725]: 2024-04-25 11:41:36.848047956 +0000 UTC m=+0.018598936 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:41:37.191 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:41:36 smithi083 podman[71725]: 2024-04-25 11:41:36.960595784 +0000 UTC m=+0.131146756 container create 6680a114ecc089f49c922d24ba646d89ec25297b2c58c79caabd052b0628bf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-a, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T11:41:37.511 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:41:37 smithi083 podman[71725]: 2024-04-25 11:41:37.204960597 +0000 UTC m=+0.375511567 container init 6680a114ecc089f49c922d24ba646d89ec25297b2c58c79caabd052b0628bf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:41:37.511 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:41:37 smithi083 podman[71725]: 2024-04-25 11:41:37.208572403 +0000 UTC m=+0.379123372 container start 6680a114ecc089f49c922d24ba646d89ec25297b2c58c79caabd052b0628bf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:41:37.511 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:41:37 smithi083 ceph-mds[71746]: -- 172.21.15.83:0/1477607443 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a9d10c9a40 con 0x55a9d109cc00 2024-04-25T11:41:37.511 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:41:37 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-a[71742]: starting mds.a at 2024-04-25T11:41:37.512 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:41:37 smithi083 bash[71725]: 6680a114ecc089f49c922d24ba646d89ec25297b2c58c79caabd052b0628bf19 2024-04-25T11:41:37.512 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:41:37 smithi083 systemd[1]: Started Ceph mds.a for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:41:37.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:37 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2528971956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:37.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:37 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2528971956' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:38.544 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:38.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:38 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:38.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:38 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:38.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:38 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:38.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:38 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:38.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:38 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:38.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:38 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:38.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:38 smithi083 ceph-mon[29330]: Deploying daemon mds.b on smithi094 2024-04-25T11:41:38.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:38 smithi083 ceph-mon[29330]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:38.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:38 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.83:6840/1576544313,v1:172.21.15.83:6841/1576544313] up:boot 2024-04-25T11:41:38.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:38 smithi083 ceph-mon[29330]: fsmap 11 up:standby 2024-04-25T11:41:38.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:38 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T11:41:38.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.671+0000 7fc4bcc49640 1 -- 172.21.15.83:0/4186731366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b81029b0 msgr2=0x7fc4b81092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.671+0000 7fc4bcc49640 1 --2- 172.21.15.83:0/4186731366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b81029b0 0x7fc4b81092d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009ca0 tx=0x7fc4b002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.671+0000 7fc4bcc49640 1 -- 172.21.15.83:0/4186731366 shutdown_connections 2024-04-25T11:41:38.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.671+0000 7fc4bcc49640 1 --2- 172.21.15.83:0/4186731366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b81029b0 0x7fc4b81092d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.672+0000 7fc4bcc49640 1 --2- 172.21.15.83:0/4186731366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4b8101ff0 0x7fc4b8102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.672+0000 7fc4bcc49640 1 --2- 172.21.15.83:0/4186731366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4b8100ea0 0x7fc4b81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.672+0000 7fc4bcc49640 1 -- 172.21.15.83:0/4186731366 >> 172.21.15.83:0/4186731366 conn(0x7fc4b80fc430 msgr2=0x7fc4b80fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:38.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.672+0000 7fc4bcc49640 1 -- 172.21.15.83:0/4186731366 shutdown_connections 2024-04-25T11:41:38.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.672+0000 7fc4bcc49640 1 -- 172.21.15.83:0/4186731366 wait complete. 2024-04-25T11:41:38.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.672+0000 7fc4bcc49640 1 Processor -- start 2024-04-25T11:41:38.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.672+0000 7fc4bcc49640 1 -- start start 2024-04-25T11:41:38.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.672+0000 7fc4bcc49640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4b8100ea0 0x7fc4b81970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.672+0000 7fc4bcc49640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4b8101ff0 0x7fc4b81975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.672+0000 7fc4bcc49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b81029b0 0x7fc4b819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4bcc49640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b810c9f0 con 0x7fc4b81029b0 2024-04-25T11:41:38.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4bcc49640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b810c870 con 0x7fc4b8100ea0 2024-04-25T11:41:38.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4bcc49640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b810cb70 con 0x7fc4b8101ff0 2024-04-25T11:41:38.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4b6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b81029b0 0x7fc4b819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:38.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4b6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b81029b0 0x7fc4b819e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43944/0 (socket says 172.21.15.83:43944) 2024-04-25T11:41:38.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4b6d9d640 1 -- 172.21.15.83:0/3263377563 learned_addr learned my addr 172.21.15.83:0/3263377563 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:38.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4b659c640 1 --2- 172.21.15.83:0/3263377563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4b8100ea0 0x7fc4b81970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:38.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4b5d9b640 1 --2- 172.21.15.83:0/3263377563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4b8101ff0 0x7fc4b81975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:38.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4b6d9d640 1 -- 172.21.15.83:0/3263377563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4b8101ff0 msgr2=0x7fc4b81975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4b6d9d640 1 --2- 172.21.15.83:0/3263377563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4b8101ff0 0x7fc4b81975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4b6d9d640 1 -- 172.21.15.83:0/3263377563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4b8100ea0 msgr2=0x7fc4b81970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4b6d9d640 1 --2- 172.21.15.83:0/3263377563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4b8100ea0 0x7fc4b81970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4b6d9d640 1 -- 172.21.15.83:0/3263377563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b819ecc0 con 0x7fc4b81029b0 2024-04-25T11:41:38.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4b6d9d640 1 --2- 172.21.15.83:0/3263377563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b81029b0 0x7fc4b819e650 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00097b0 tx=0x7fc4b0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:38.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4a77fe640 1 -- 172.21.15.83:0/3263377563 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0004240 con 0x7fc4b81029b0 2024-04-25T11:41:38.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4a77fe640 1 -- 172.21.15.83:0/3263377563 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b00317c0 con 0x7fc4b81029b0 2024-04-25T11:41:38.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4a77fe640 1 -- 172.21.15.83:0/3263377563 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b002fd80 con 0x7fc4b81029b0 2024-04-25T11:41:38.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4bcc49640 1 -- 172.21.15.83:0/3263377563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b819ef50 con 0x7fc4b81029b0 2024-04-25T11:41:38.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.673+0000 7fc4bcc49640 1 -- 172.21.15.83:0/3263377563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b819f3c0 con 0x7fc4b81029b0 2024-04-25T11:41:38.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.674+0000 7fc4bcc49640 1 -- 172.21.15.83:0/3263377563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b81081b0 con 0x7fc4b81029b0 2024-04-25T11:41:38.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.676+0000 7fc4a77fe640 1 -- 172.21.15.83:0/3263377563 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc4b00043e0 con 0x7fc4b81029b0 2024-04-25T11:41:38.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.676+0000 7fc4a77fe640 1 --2- 172.21.15.83:0/3263377563 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc4a00a3c30 0x7fc4a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:38.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.676+0000 7fc4b659c640 1 --2- 172.21.15.83:0/3263377563 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc4a00a3c30 0x7fc4a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:38.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.676+0000 7fc4a77fe640 1 -- 172.21.15.83:0/3263377563 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc4b00f1860 con 0x7fc4b81029b0 2024-04-25T11:41:38.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.677+0000 7fc4b659c640 1 --2- 172.21.15.83:0/3263377563 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc4a00a3c30 0x7fc4a00a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8005d00 tx=0x7fc4a800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:38.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.680+0000 7fc4a77fe640 1 -- 172.21.15.83:0/3263377563 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc4b00be800 con 0x7fc4b81029b0 2024-04-25T11:41:38.681 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:38 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:38.681 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:38 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:38.681 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:38 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:38.682 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:38 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:38.682 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:38 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:38.682 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:38 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:38.682 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:38 smithi094 ceph-mon[33032]: Deploying daemon mds.b on smithi094 2024-04-25T11:41:38.682 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:38 smithi094 ceph-mon[33032]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:38.682 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:38 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.83:6840/1576544313,v1:172.21.15.83:6841/1576544313] up:boot 2024-04-25T11:41:38.682 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:38 smithi094 ceph-mon[33032]: fsmap 11 up:standby 2024-04-25T11:41:38.682 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:38 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T11:41:38.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:38 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:38.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:38 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:38.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:38 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:38.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:38 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T11:41:38.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:38 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T11:41:38.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:38 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:38.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:38 smithi111 ceph-mon[32784]: Deploying daemon mds.b on smithi094 2024-04-25T11:41:38.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:38 smithi111 ceph-mon[32784]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:38.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:38 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.83:6840/1576544313,v1:172.21.15.83:6841/1576544313] up:boot 2024-04-25T11:41:38.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:38 smithi111 ceph-mon[32784]: fsmap 11 up:standby 2024-04-25T11:41:38.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:38 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T11:41:38.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.898+0000 7fc4bcc49640 1 -- 172.21.15.83:0/3263377563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4b802d5c0 con 0x7fc4b81029b0 2024-04-25T11:41:38.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.898+0000 7fc4a77fe640 1 -- 172.21.15.83:0/3263377563 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7fc4b00be100 con 0x7fc4b81029b0 2024-04-25T11:41:38.885 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:38.885 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1568064835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1568064835},{"type":"v1","addr":"172.21.15.83:6839","nonce":1568064835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1576544313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1576544313},{"type":"v1","addr":"172.21.15.83:6841","nonce":1576544313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2475976592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2475976592},{"type":"v1","addr":"172.21.15.94:6837","nonce":2475976592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2927430878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2927430878},{"type":"v1","addr":"172.21.15.111:6837","nonce":2927430878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/2233531151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2233531151},{"type":"v1","addr":"172.21.15.111:6839","nonce":2233531151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T11:41:38.885 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 12 2024-04-25T11:41:38.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.901+0000 7fc4bcc49640 1 -- 172.21.15.83:0/3263377563 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc4a00a3c30 msgr2=0x7fc4a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.901+0000 7fc4bcc49640 1 --2- 172.21.15.83:0/3263377563 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc4a00a3c30 0x7fc4a00a60f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8005d00 tx=0x7fc4a800b410 comp rx=0 tx=0).stop 2024-04-25T11:41:38.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.901+0000 7fc4bcc49640 1 -- 172.21.15.83:0/3263377563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b81029b0 msgr2=0x7fc4b819e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:38.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.901+0000 7fc4bcc49640 1 --2- 172.21.15.83:0/3263377563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b81029b0 0x7fc4b819e650 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00097b0 tx=0x7fc4b0003c20 comp rx=0 tx=0).stop 2024-04-25T11:41:38.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.901+0000 7fc4bcc49640 1 -- 172.21.15.83:0/3263377563 shutdown_connections 2024-04-25T11:41:38.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.901+0000 7fc4bcc49640 1 --2- 172.21.15.83:0/3263377563 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc4a00a3c30 0x7fc4a00a60f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.901+0000 7fc4bcc49640 1 --2- 172.21.15.83:0/3263377563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4b81029b0 0x7fc4b819e650 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.901+0000 7fc4bcc49640 1 --2- 172.21.15.83:0/3263377563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4b8101ff0 0x7fc4b81975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.901+0000 7fc4bcc49640 1 --2- 172.21.15.83:0/3263377563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4b8100ea0 0x7fc4b81970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:38.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.901+0000 7fc4bcc49640 1 -- 172.21.15.83:0/3263377563 >> 172.21.15.83:0/3263377563 conn(0x7fc4b80fc430 msgr2=0x7fc4b8105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:38.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.901+0000 7fc4bcc49640 1 -- 172.21.15.83:0/3263377563 shutdown_connections 2024-04-25T11:41:38.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:38.901+0000 7fc4bcc49640 1 -- 172.21.15.83:0/3263377563 wait complete. 2024-04-25T11:41:39.574 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:39 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3263377563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:39.574 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:41:39 smithi094 systemd[1]: Starting Ceph mds.b for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:41:39.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:39 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3263377563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:39.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:39 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3263377563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:40.056 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:41:39 smithi094 podman[63270]: 2024-04-25 11:41:39.707198249 +0000 UTC m=+0.018655407 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:41:40.056 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:41:39 smithi094 podman[63270]: 2024-04-25 11:41:39.816050947 +0000 UTC m=+0.127508129 container create abf805b5081f4b5f242f708566620d79713abbe5b0948d3c65f31f8a419cc943 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T11:41:40.056 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:41:40 smithi094 podman[63270]: 2024-04-25 11:41:40.043906283 +0000 UTC m=+0.355363427 container init abf805b5081f4b5f242f708566620d79713abbe5b0948d3c65f31f8a419cc943 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T11:41:40.056 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:41:40 smithi094 podman[63270]: 2024-04-25 11:41:40.046427473 +0000 UTC m=+0.357884616 container start abf805b5081f4b5f242f708566620d79713abbe5b0948d3c65f31f8a419cc943 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-04-25T11:41:40.056 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:41:40 smithi094 ceph-mds[63289]: -- 172.21.15.94:0/672808866 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557ce62afa40 con 0x557ce6282c00 2024-04-25T11:41:40.404 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:41:40 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b[63285]: starting mds.b at 2024-04-25T11:41:40.405 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:41:40 smithi094 bash[63270]: abf805b5081f4b5f242f708566620d79713abbe5b0948d3c65f31f8a419cc943 2024-04-25T11:41:40.405 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:41:40 smithi094 systemd[1]: Started Ceph mds.b for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:41:40.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:40 smithi083 ceph-mon[29330]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:40.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:40.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:40.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:40.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:40.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:40 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:40.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:40 smithi094 ceph-mon[33032]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:40.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:40 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:40.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:40 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:40.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:40 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:40.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:40 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:40.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:40 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:40.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:40 smithi111 ceph-mon[32784]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:40.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:40.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:40.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:40.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:40.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:40 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:40.899 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:41.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.028+0000 7f701fb08640 1 -- 172.21.15.83:0/475302723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7018102040 msgr2=0x7f70181024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.028+0000 7f701fb08640 1 --2- 172.21.15.83:0/475302723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7018102040 0x7f70181024c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7008009900 tx=0x7f700802a410 comp rx=0 tx=0).stop 2024-04-25T11:41:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.028+0000 7f701fb08640 1 -- 172.21.15.83:0/475302723 shutdown_connections 2024-04-25T11:41:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.028+0000 7f701fb08640 1 --2- 172.21.15.83:0/475302723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7018102040 0x7f70181024c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.028+0000 7f701fb08640 1 --2- 172.21.15.83:0/475302723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7018102a00 0x7f7018109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.028+0000 7f701fb08640 1 --2- 172.21.15.83:0/475302723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7018100e60 0x7f7018101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.028+0000 7f701fb08640 1 -- 172.21.15.83:0/475302723 >> 172.21.15.83:0/475302723 conn(0x7f70180fc440 msgr2=0x7f70180fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.028+0000 7f701fb08640 1 -- 172.21.15.83:0/475302723 shutdown_connections 2024-04-25T11:41:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.028+0000 7f701fb08640 1 -- 172.21.15.83:0/475302723 wait complete. 2024-04-25T11:41:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701fb08640 1 Processor -- start 2024-04-25T11:41:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701fb08640 1 -- start start 2024-04-25T11:41:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701fb08640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7018100e60 0x7f701806b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701fb08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7018102040 0x7f701819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701fb08640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7018102a00 0x7f701819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701fb08640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f701810ca00 con 0x7f7018102a00 2024-04-25T11:41:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701fb08640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f701810c880 con 0x7f7018102040 2024-04-25T11:41:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701fb08640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f701810cb80 con 0x7f7018100e60 2024-04-25T11:41:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701e0a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7018102a00 0x7f701819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701e0a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7018102a00 0x7f701819f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43962/0 (socket says 172.21.15.83:43962) 2024-04-25T11:41:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701e0a5640 1 -- 172.21.15.83:0/756222915 learned_addr learned my addr 172.21.15.83:0/756222915 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701d8a4640 1 --2- 172.21.15.83:0/756222915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7018100e60 0x7f701806b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701d0a3640 1 --2- 172.21.15.83:0/756222915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7018102040 0x7f701819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701e0a5640 1 -- 172.21.15.83:0/756222915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7018100e60 msgr2=0x7f701806b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701e0a5640 1 --2- 172.21.15.83:0/756222915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7018100e60 0x7f701806b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701e0a5640 1 -- 172.21.15.83:0/756222915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7018102040 msgr2=0x7f701819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701e0a5640 1 --2- 172.21.15.83:0/756222915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7018102040 0x7f701819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.029+0000 7f701e0a5640 1 -- 172.21.15.83:0/756222915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701819fe00 con 0x7f7018102a00 2024-04-25T11:41:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.030+0000 7f701e0a5640 1 --2- 172.21.15.83:0/756222915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7018102a00 0x7f701819f540 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f701000eb10 tx=0x7f701000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.030+0000 7f700effd640 1 -- 172.21.15.83:0/756222915 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010004b20 con 0x7f7018102a00 2024-04-25T11:41:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.030+0000 7f700effd640 1 -- 172.21.15.83:0/756222915 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7010004cc0 con 0x7f7018102a00 2024-04-25T11:41:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.030+0000 7f701fb08640 1 -- 172.21.15.83:0/756222915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70181a00f0 con 0x7f7018102a00 2024-04-25T11:41:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.030+0000 7f700effd640 1 -- 172.21.15.83:0/756222915 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010010670 con 0x7f7018102a00 2024-04-25T11:41:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.030+0000 7f701fb08640 1 -- 172.21.15.83:0/756222915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7018196eb0 con 0x7f7018102a00 2024-04-25T11:41:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.031+0000 7f701fb08640 1 -- 172.21.15.83:0/756222915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70181082a0 con 0x7f7018102a00 2024-04-25T11:41:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.032+0000 7f700effd640 1 -- 172.21.15.83:0/756222915 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7010005000 con 0x7f7018102a00 2024-04-25T11:41:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.032+0000 7f700effd640 1 --2- 172.21.15.83:0/756222915 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f70040a3ba0 0x7f70040a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.033+0000 7f701d8a4640 1 --2- 172.21.15.83:0/756222915 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f70040a3ba0 0x7f70040a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.033+0000 7f700effd640 1 -- 172.21.15.83:0/756222915 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f70100d0690 con 0x7f7018102a00 2024-04-25T11:41:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.033+0000 7f701d8a4640 1 --2- 172.21.15.83:0/756222915 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f70040a3ba0 0x7f70040a6060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7018101bb0 tx=0x7f7014005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:41.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.037+0000 7f700effd640 1 -- 172.21.15.83:0/756222915 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7010053ff0 con 0x7f7018102a00 2024-04-25T11:41:41.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.254+0000 7f701fb08640 1 -- 172.21.15.83:0/756222915 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f701802d5c0 con 0x7f7018102a00 2024-04-25T11:41:41.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.255+0000 7f700effd640 1 -- 172.21.15.83:0/756222915 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10485 (secure 0 0 0) 0x7f701009d510 con 0x7f7018102a00 2024-04-25T11:41:41.242 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:41.242 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/1568064835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1568064835},{"type":"v1","addr":"172.21.15.83:6839","nonce":1568064835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/1576544313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1576544313},{"type":"v1","addr":"172.21.15.83:6841","nonce":1576544313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24416,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2475976592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2475976592},{"type":"v1","addr":"172.21.15.94:6837","nonce":2475976592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/2927430878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2927430878},{"type":"v1","addr":"172.21.15.111:6837","nonce":2927430878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/2233531151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2233531151},{"type":"v1","addr":"172.21.15.111:6839","nonce":2233531151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/1461078471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1461078471},{"type":"v1","addr":"172.21.15.94:6839","nonce":1461078471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T11:41:41.242 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 13 2024-04-25T11:41:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.257+0000 7f701fb08640 1 -- 172.21.15.83:0/756222915 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f70040a3ba0 msgr2=0x7f70040a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.257+0000 7f701fb08640 1 --2- 172.21.15.83:0/756222915 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f70040a3ba0 0x7f70040a6060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7018101bb0 tx=0x7f7014005f90 comp rx=0 tx=0).stop 2024-04-25T11:41:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.257+0000 7f701fb08640 1 -- 172.21.15.83:0/756222915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7018102a00 msgr2=0x7f701819f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.257+0000 7f701fb08640 1 --2- 172.21.15.83:0/756222915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7018102a00 0x7f701819f540 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f701000eb10 tx=0x7f701000c2d0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.257+0000 7f701fb08640 1 -- 172.21.15.83:0/756222915 shutdown_connections 2024-04-25T11:41:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.257+0000 7f701fb08640 1 --2- 172.21.15.83:0/756222915 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f70040a3ba0 0x7f70040a6060 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7018101bb0 tx=0x7f7014005f90 comp rx=0 tx=0).stop 2024-04-25T11:41:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.257+0000 7f701fb08640 1 --2- 172.21.15.83:0/756222915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7018102a00 0x7f701819f540 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.257+0000 7f701fb08640 1 --2- 172.21.15.83:0/756222915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7018102040 0x7f701819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.257+0000 7f701fb08640 1 --2- 172.21.15.83:0/756222915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7018100e60 0x7f701806b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.257+0000 7f701fb08640 1 -- 172.21.15.83:0/756222915 >> 172.21.15.83:0/756222915 conn(0x7f70180fc440 msgr2=0x7f7018105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.258+0000 7f701fb08640 1 -- 172.21.15.83:0/756222915 shutdown_connections 2024-04-25T11:41:41.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.258+0000 7f701fb08640 1 -- 172.21.15.83:0/756222915 wait complete. 2024-04-25T11:41:41.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:41:41.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.373+0000 7f81d15cd640 1 -- 172.21.15.83:0/1047742085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81cc1053e0 msgr2=0x7f81cc1077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.373+0000 7f81d15cd640 1 --2- 172.21.15.83:0/1047742085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81cc1053e0 0x7f81cc1077d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f81c0009900 tx=0x7f81c002a410 comp rx=0 tx=0).stop 2024-04-25T11:41:41.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.373+0000 7f81d15cd640 1 -- 172.21.15.83:0/1047742085 shutdown_connections 2024-04-25T11:41:41.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.373+0000 7f81d15cd640 1 --2- 172.21.15.83:0/1047742085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81cc1053e0 0x7f81cc1077d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.373+0000 7f81d15cd640 1 --2- 172.21.15.83:0/1047742085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81cc107d10 0x7f81cc10a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.373+0000 7f81d15cd640 1 --2- 172.21.15.83:0/1047742085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81cc102ab0 0x7f81cc104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.373+0000 7f81d15cd640 1 -- 172.21.15.83:0/1047742085 >> 172.21.15.83:0/1047742085 conn(0x7f81cc0fc440 msgr2=0x7f81cc0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:41.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.373+0000 7f81d15cd640 1 -- 172.21.15.83:0/1047742085 shutdown_connections 2024-04-25T11:41:41.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.373+0000 7f81d15cd640 1 -- 172.21.15.83:0/1047742085 wait complete. 2024-04-25T11:41:41.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81d15cd640 1 Processor -- start 2024-04-25T11:41:41.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81d15cd640 1 -- start start 2024-04-25T11:41:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81d15cd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81cc102ab0 0x7f81cc198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81d15cd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81cc1053e0 0x7f81cc1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81d15cd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81cc107d10 0x7f81cc19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81d15cd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc10db10 con 0x7f81cc107d10 2024-04-25T11:41:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81d15cd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc10d990 con 0x7f81cc1053e0 2024-04-25T11:41:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81d15cd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc10dc90 con 0x7f81cc102ab0 2024-04-25T11:41:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81cc107d10 0x7f81cc19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81ca7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81cc1053e0 0x7f81cc1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81cc107d10 0x7f81cc19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43986/0 (socket says 172.21.15.83:43986) 2024-04-25T11:41:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81cb7fe640 1 -- 172.21.15.83:0/434887448 learned_addr learned my addr 172.21.15.83:0/434887448 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81caffd640 1 --2- 172.21.15.83:0/434887448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81cc102ab0 0x7f81cc198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81ca7fc640 1 -- 172.21.15.83:0/434887448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81cc102ab0 msgr2=0x7f81cc198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81ca7fc640 1 --2- 172.21.15.83:0/434887448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81cc102ab0 0x7f81cc198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81ca7fc640 1 -- 172.21.15.83:0/434887448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81cc107d10 msgr2=0x7f81cc19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81ca7fc640 1 --2- 172.21.15.83:0/434887448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81cc107d10 0x7f81cc19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.374+0000 7f81ca7fc640 1 -- 172.21.15.83:0/434887448 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc19fe90 con 0x7f81cc1053e0 2024-04-25T11:41:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.375+0000 7f81ca7fc640 1 --2- 172.21.15.83:0/434887448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81cc1053e0 0x7f81cc1987b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f81c0006010 tx=0x7f81c00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.375+0000 7f81b3fff640 1 -- 172.21.15.83:0/434887448 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c003f030 con 0x7f81cc1053e0 2024-04-25T11:41:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.375+0000 7f81b3fff640 1 -- 172.21.15.83:0/434887448 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c0031910 con 0x7f81cc1053e0 2024-04-25T11:41:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.375+0000 7f81d15cd640 1 -- 172.21.15.83:0/434887448 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81cc1a0120 con 0x7f81cc1053e0 2024-04-25T11:41:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.375+0000 7f81b3fff640 1 -- 172.21.15.83:0/434887448 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0035070 con 0x7f81cc1053e0 2024-04-25T11:41:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.375+0000 7f81d15cd640 1 -- 172.21.15.83:0/434887448 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81cc1a0610 con 0x7f81cc1053e0 2024-04-25T11:41:41.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.376+0000 7f81d15cd640 1 -- 172.21.15.83:0/434887448 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81cc0fdf30 con 0x7f81cc1053e0 2024-04-25T11:41:41.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.377+0000 7f81b3fff640 1 -- 172.21.15.83:0/434887448 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f81c0042030 con 0x7f81cc1053e0 2024-04-25T11:41:41.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.377+0000 7f81b3fff640 1 --2- 172.21.15.83:0/434887448 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f81b40a3bb0 0x7f81b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.378+0000 7f81caffd640 1 --2- 172.21.15.83:0/434887448 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f81b40a3bb0 0x7f81b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.378+0000 7f81b3fff640 1 -- 172.21.15.83:0/434887448 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f81c00f0600 con 0x7f81cc1053e0 2024-04-25T11:41:41.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.378+0000 7f81caffd640 1 --2- 172.21.15.83:0/434887448 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f81b40a3bb0 0x7f81b40a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009bd0 tx=0x7f81bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.381+0000 7f81b3fff640 1 -- 172.21.15.83:0/434887448 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81c00bce10 con 0x7f81cc1053e0 2024-04-25T11:41:41.511 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:41 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] up:boot 2024-04-25T11:41:41.511 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:41 smithi083 ceph-mon[29330]: fsmap 12 up:standby 2024-04-25T11:41:41.511 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:41 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:41:41.511 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:41 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:41:41.511 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:41 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/756222915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:41.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:41 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] up:boot 2024-04-25T11:41:41.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:41 smithi094 ceph-mon[33032]: fsmap 12 up:standby 2024-04-25T11:41:41.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:41 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:41:41.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:41 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:41:41.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:41 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/756222915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:41.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.568+0000 7f81d15cd640 1 -- 172.21.15.83:0/434887448 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81cc02d810 con 0x7f81cc1053e0 2024-04-25T11:41:41.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.570+0000 7f81b3fff640 1 -- 172.21.15.83:0/434887448 <== mon.2 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18384 (secure 0 0 0) 0x7f81c00c1110 con 0x7f81cc1053e0 2024-04-25T11:41:41.556 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:41.557 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":82,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:40:58.967556+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:41:41.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.572+0000 7f81d15cd640 1 -- 172.21.15.83:0/434887448 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f81b40a3bb0 msgr2=0x7f81b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.572+0000 7f81d15cd640 1 --2- 172.21.15.83:0/434887448 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f81b40a3bb0 0x7f81b40a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f81bc009bd0 tx=0x7f81bc005f90 comp rx=0 tx=0).stop 2024-04-25T11:41:41.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.572+0000 7f81d15cd640 1 -- 172.21.15.83:0/434887448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81cc1053e0 msgr2=0x7f81cc1987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.572+0000 7f81d15cd640 1 --2- 172.21.15.83:0/434887448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81cc1053e0 0x7f81cc1987b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f81c0006010 tx=0x7f81c00042f0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.572+0000 7f81d15cd640 1 -- 172.21.15.83:0/434887448 shutdown_connections 2024-04-25T11:41:41.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.572+0000 7f81d15cd640 1 --2- 172.21.15.83:0/434887448 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f81b40a3bb0 0x7f81b40a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.572+0000 7f81d15cd640 1 --2- 172.21.15.83:0/434887448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81cc107d10 0x7f81cc19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.572+0000 7f81d15cd640 1 --2- 172.21.15.83:0/434887448 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81cc1053e0 0x7f81cc1987b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.572+0000 7f81d15cd640 1 --2- 172.21.15.83:0/434887448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81cc102ab0 0x7f81cc198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.572+0000 7f81d15cd640 1 -- 172.21.15.83:0/434887448 >> 172.21.15.83:0/434887448 conn(0x7f81cc0fc440 msgr2=0x7f81cc104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:41.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.572+0000 7f81d15cd640 1 -- 172.21.15.83:0/434887448 shutdown_connections 2024-04-25T11:41:41.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.572+0000 7f81d15cd640 1 -- 172.21.15.83:0/434887448 wait complete. 2024-04-25T11:41:41.570 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:37:15.914779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:41:41.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:41:41.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:41 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] up:boot 2024-04-25T11:41:41.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:41 smithi111 ceph-mon[32784]: fsmap 12 up:standby 2024-04-25T11:41:41.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:41 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:41:41.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:41 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:41:41.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:41 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/756222915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.684+0000 7fa1f00cb640 1 -- 172.21.15.83:0/2139772948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e8102440 msgr2=0x7fa1e8102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.684+0000 7fa1f00cb640 1 --2- 172.21.15.83:0/2139772948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e8102440 0x7fa1e8102860 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009900 tx=0x7fa1e402a410 comp rx=0 tx=0).stop 2024-04-25T11:41:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.684+0000 7fa1f00cb640 1 -- 172.21.15.83:0/2139772948 shutdown_connections 2024-04-25T11:41:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.684+0000 7fa1f00cb640 1 --2- 172.21.15.83:0/2139772948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1e8101b30 0x7fa1e81093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.684+0000 7fa1f00cb640 1 --2- 172.21.15.83:0/2139772948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1e8101170 0x7fa1e81015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.684+0000 7fa1f00cb640 1 --2- 172.21.15.83:0/2139772948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e8102440 0x7fa1e8102860 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.684+0000 7fa1f00cb640 1 -- 172.21.15.83:0/2139772948 >> 172.21.15.83:0/2139772948 conn(0x7fa1e80fbb80 msgr2=0x7fa1e80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.685+0000 7fa1f00cb640 1 -- 172.21.15.83:0/2139772948 shutdown_connections 2024-04-25T11:41:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.685+0000 7fa1f00cb640 1 -- 172.21.15.83:0/2139772948 wait complete. 2024-04-25T11:41:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.685+0000 7fa1f00cb640 1 Processor -- start 2024-04-25T11:41:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.685+0000 7fa1f00cb640 1 -- start start 2024-04-25T11:41:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.685+0000 7fa1f00cb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1e8101170 0x7fa1e8197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.685+0000 7fa1f00cb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e8101b30 0x7fa1e8197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.685+0000 7fa1f00cb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1e8102440 0x7fa1e819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.685+0000 7fa1f00cb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e810c9e0 con 0x7fa1e8101b30 2024-04-25T11:41:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.685+0000 7fa1f00cb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e810c860 con 0x7fa1e8101170 2024-04-25T11:41:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.685+0000 7fa1f00cb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e810cb60 con 0x7fa1e8102440 2024-04-25T11:41:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.685+0000 7fa1ed666640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e8101b30 0x7fa1e8197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.685+0000 7fa1ed666640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e8101b30 0x7fa1e8197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44000/0 (socket says 172.21.15.83:44000) 2024-04-25T11:41:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.685+0000 7fa1ed666640 1 -- 172.21.15.83:0/3287668873 learned_addr learned my addr 172.21.15.83:0/3287668873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.685+0000 7fa1ee668640 1 --2- 172.21.15.83:0/3287668873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1e8102440 0x7fa1e819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.685+0000 7fa1ede67640 1 --2- 172.21.15.83:0/3287668873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1e8101170 0x7fa1e8197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.686+0000 7fa1ed666640 1 -- 172.21.15.83:0/3287668873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1e8102440 msgr2=0x7fa1e819e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.686+0000 7fa1ed666640 1 --2- 172.21.15.83:0/3287668873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1e8102440 0x7fa1e819e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.686+0000 7fa1ed666640 1 -- 172.21.15.83:0/3287668873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1e8101170 msgr2=0x7fa1e8197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.686+0000 7fa1ed666640 1 --2- 172.21.15.83:0/3287668873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1e8101170 0x7fa1e8197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.686+0000 7fa1ed666640 1 -- 172.21.15.83:0/3287668873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e819ed50 con 0x7fa1e8101b30 2024-04-25T11:41:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.686+0000 7fa1ed666640 1 --2- 172.21.15.83:0/3287668873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e8101b30 0x7fa1e8197670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800d950 tx=0x7fa1d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.686+0000 7fa1deffd640 1 -- 172.21.15.83:0/3287668873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8017030 con 0x7fa1e8101b30 2024-04-25T11:41:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.686+0000 7fa1deffd640 1 -- 172.21.15.83:0/3287668873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1d8004020 con 0x7fa1e8101b30 2024-04-25T11:41:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.686+0000 7fa1f00cb640 1 -- 172.21.15.83:0/3287668873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e819efe0 con 0x7fa1e8101b30 2024-04-25T11:41:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.686+0000 7fa1deffd640 1 -- 172.21.15.83:0/3287668873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80047a0 con 0x7fa1e8101b30 2024-04-25T11:41:41.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.686+0000 7fa1f00cb640 1 -- 172.21.15.83:0/3287668873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e819f550 con 0x7fa1e8101b30 2024-04-25T11:41:41.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.687+0000 7fa1f00cb640 1 -- 172.21.15.83:0/3287668873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e8108370 con 0x7fa1e8101b30 2024-04-25T11:41:41.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.688+0000 7fa1deffd640 1 -- 172.21.15.83:0/3287668873 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa1d8009a30 con 0x7fa1e8101b30 2024-04-25T11:41:41.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.689+0000 7fa1deffd640 1 --2- 172.21.15.83:0/3287668873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa1d40a3c30 0x7fa1d40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.689+0000 7fa1ede67640 1 --2- 172.21.15.83:0/3287668873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa1d40a3c30 0x7fa1d40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.689+0000 7fa1deffd640 1 -- 172.21.15.83:0/3287668873 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa1d80d00b0 con 0x7fa1e8101b30 2024-04-25T11:41:41.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.689+0000 7fa1ede67640 1 --2- 172.21.15.83:0/3287668873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa1d40a3c30 0x7fa1d40a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4000c00 tx=0x7fa1e40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:41.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.693+0000 7fa1deffd640 1 -- 172.21.15.83:0/3287668873 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1d809cfb0 con 0x7fa1e8101b30 2024-04-25T11:41:41.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.890+0000 7fa1f00cb640 1 -- 172.21.15.83:0/3287668873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1e802d810 con 0x7fa1e8101b30 2024-04-25T11:41:41.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.890+0000 7fa1deffd640 1 -- 172.21.15.83:0/3287668873 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fa1d809c8b0 con 0x7fa1e8101b30 2024-04-25T11:41:41.877 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T11:41:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.893+0000 7fa1f00cb640 1 -- 172.21.15.83:0/3287668873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa1d40a3c30 msgr2=0x7fa1d40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.893+0000 7fa1f00cb640 1 --2- 172.21.15.83:0/3287668873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa1d40a3c30 0x7fa1d40a60f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4000c00 tx=0x7fa1e40314a0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.893+0000 7fa1f00cb640 1 -- 172.21.15.83:0/3287668873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e8101b30 msgr2=0x7fa1e8197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.893+0000 7fa1f00cb640 1 --2- 172.21.15.83:0/3287668873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e8101b30 0x7fa1e8197670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800d950 tx=0x7fa1d800de10 comp rx=0 tx=0).stop 2024-04-25T11:41:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.893+0000 7fa1f00cb640 1 -- 172.21.15.83:0/3287668873 shutdown_connections 2024-04-25T11:41:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.893+0000 7fa1f00cb640 1 --2- 172.21.15.83:0/3287668873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa1d40a3c30 0x7fa1d40a60f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.893+0000 7fa1f00cb640 1 --2- 172.21.15.83:0/3287668873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e8101b30 0x7fa1e8197670 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.893+0000 7fa1f00cb640 1 --2- 172.21.15.83:0/3287668873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1e8102440 0x7fa1e819e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.894+0000 7fa1f00cb640 1 --2- 172.21.15.83:0/3287668873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa1e8101170 0x7fa1e8197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.894+0000 7fa1f00cb640 1 -- 172.21.15.83:0/3287668873 >> 172.21.15.83:0/3287668873 conn(0x7fa1e80fbb80 msgr2=0x7fa1e8105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.894+0000 7fa1f00cb640 1 -- 172.21.15.83:0/3287668873 shutdown_connections 2024-04-25T11:41:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:41.894+0000 7fa1f00cb640 1 -- 172.21.15.83:0/3287668873 wait complete. 2024-04-25T11:41:41.891 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:41:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.009+0000 7f1569810640 1 -- 172.21.15.83:0/718052779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564101a30 msgr2=0x7f15641092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.009+0000 7f1569810640 1 --2- 172.21.15.83:0/718052779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564101a30 0x7f15641092f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f155c009ca0 tx=0x7f155c02a8c0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f1569810640 1 -- 172.21.15.83:0/718052779 shutdown_connections 2024-04-25T11:41:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f1569810640 1 --2- 172.21.15.83:0/718052779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564101a30 0x7f15641092f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f1569810640 1 --2- 172.21.15.83:0/718052779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1564101090 0x7f15641014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f1569810640 1 --2- 172.21.15.83:0/718052779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1564102440 0x7f1564102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:41.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f1569810640 1 -- 172.21.15.83:0/718052779 >> 172.21.15.83:0/718052779 conn(0x7f15640fbb80 msgr2=0x7f15640fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f1569810640 1 -- 172.21.15.83:0/718052779 shutdown_connections 2024-04-25T11:41:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f1569810640 1 -- 172.21.15.83:0/718052779 wait complete. 2024-04-25T11:41:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f1569810640 1 Processor -- start 2024-04-25T11:41:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f1569810640 1 -- start start 2024-04-25T11:41:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f1569810640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1564101090 0x7f1564104df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f1569810640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564101a30 0x7f1564103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f1569810640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1564102440 0x7f15641039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f1569810640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f156410c9b0 con 0x7f1564101a30 2024-04-25T11:41:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f1569810640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f156410c830 con 0x7f1564102440 2024-04-25T11:41:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f1569810640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f156410cb30 con 0x7f1564101090 2024-04-25T11:41:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f155bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564101a30 0x7f1564103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.010+0000 7f155bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564101a30 0x7f1564103480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44028/0 (socket says 172.21.15.83:44028) 2024-04-25T11:41:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.011+0000 7f155bfff640 1 -- 172.21.15.83:0/1350522599 learned_addr learned my addr 172.21.15.83:0/1350522599 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.011+0000 7f1562ffd640 1 --2- 172.21.15.83:0/1350522599 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1564101090 0x7f1564104df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:41.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.011+0000 7f15637fe640 1 --2- 172.21.15.83:0/1350522599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1564102440 0x7f15641039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.011+0000 7f155bfff640 1 -- 172.21.15.83:0/1350522599 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1564101090 msgr2=0x7f1564104df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.011+0000 7f155bfff640 1 --2- 172.21.15.83:0/1350522599 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1564101090 0x7f1564104df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.011+0000 7f155bfff640 1 -- 172.21.15.83:0/1350522599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1564102440 msgr2=0x7f15641039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.011+0000 7f155bfff640 1 --2- 172.21.15.83:0/1350522599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1564102440 0x7f15641039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.011+0000 7f155bfff640 1 -- 172.21.15.83:0/1350522599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1564104280 con 0x7f1564101a30 2024-04-25T11:41:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.011+0000 7f15637fe640 1 --2- 172.21.15.83:0/1350522599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1564102440 0x7f15641039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.011+0000 7f155bfff640 1 --2- 172.21.15.83:0/1350522599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564101a30 0x7f1564103480 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f154c00e9f0 tx=0x7f154c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.011+0000 7f156880e640 1 -- 172.21.15.83:0/1350522599 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c00cd80 con 0x7f1564101a30 2024-04-25T11:41:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.011+0000 7f1569810640 1 -- 172.21.15.83:0/1350522599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15641a22f0 con 0x7f1564101a30 2024-04-25T11:41:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.011+0000 7f156880e640 1 -- 172.21.15.83:0/1350522599 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f154c0108e0 con 0x7f1564101a30 2024-04-25T11:41:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.011+0000 7f156880e640 1 -- 172.21.15.83:0/1350522599 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c018980 con 0x7f1564101a30 2024-04-25T11:41:42.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.011+0000 7f1569810640 1 -- 172.21.15.83:0/1350522599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15641a2860 con 0x7f1564101a30 2024-04-25T11:41:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.012+0000 7f1569810640 1 -- 172.21.15.83:0/1350522599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15641a2b80 con 0x7f1564101a30 2024-04-25T11:41:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.014+0000 7f156880e640 1 -- 172.21.15.83:0/1350522599 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f154c010430 con 0x7f1564101a30 2024-04-25T11:41:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.014+0000 7f156880e640 1 --2- 172.21.15.83:0/1350522599 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f15500a3c30 0x7f15500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:42.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.014+0000 7f1562ffd640 1 --2- 172.21.15.83:0/1350522599 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f15500a3c30 0x7f15500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.014+0000 7f156880e640 1 -- 172.21.15.83:0/1350522599 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f154c0d0810 con 0x7f1564101a30 2024-04-25T11:41:42.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.015+0000 7f1562ffd640 1 --2- 172.21.15.83:0/1350522599 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f15500a3c30 0x7f15500a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1554009bd0 tx=0x7f1554005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:42.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.018+0000 7f156880e640 1 -- 172.21.15.83:0/1350522599 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f154c09d5f0 con 0x7f1564101a30 2024-04-25T11:41:42.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.228+0000 7f1569810640 1 -- 172.21.15.83:0/1350522599 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f15641a2d50 con 0x7f1564101a30 2024-04-25T11:41:42.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.229+0000 7f156880e640 1 -- 172.21.15.83:0/1350522599 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f154c09cef0 con 0x7f1564101a30 2024-04-25T11:41:42.216 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:42.216 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-25T11:41:42.216 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:41:42.216 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-25T11:41:42.216 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-25T11:41:42.216 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:41:42.216 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-25T11:41:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.231+0000 7f1569810640 1 -- 172.21.15.83:0/1350522599 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f15500a3c30 msgr2=0x7f15500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.231+0000 7f1569810640 1 --2- 172.21.15.83:0/1350522599 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f15500a3c30 0x7f15500a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1554009bd0 tx=0x7f1554005f90 comp rx=0 tx=0).stop 2024-04-25T11:41:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.231+0000 7f1569810640 1 -- 172.21.15.83:0/1350522599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564101a30 msgr2=0x7f1564103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.231+0000 7f1569810640 1 --2- 172.21.15.83:0/1350522599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564101a30 0x7f1564103480 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f154c00e9f0 tx=0x7f154c00eeb0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.231+0000 7f1569810640 1 -- 172.21.15.83:0/1350522599 shutdown_connections 2024-04-25T11:41:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.231+0000 7f1569810640 1 --2- 172.21.15.83:0/1350522599 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f15500a3c30 0x7f15500a60f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.231+0000 7f1569810640 1 --2- 172.21.15.83:0/1350522599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1564101a30 0x7f1564103480 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.231+0000 7f1569810640 1 --2- 172.21.15.83:0/1350522599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1564102440 0x7f15641039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.231+0000 7f1569810640 1 --2- 172.21.15.83:0/1350522599 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1564101090 0x7f1564104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.231+0000 7f1569810640 1 -- 172.21.15.83:0/1350522599 >> 172.21.15.83:0/1350522599 conn(0x7f15640fbb80 msgr2=0x7f15640fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.231+0000 7f1569810640 1 -- 172.21.15.83:0/1350522599 shutdown_connections 2024-04-25T11:41:42.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.231+0000 7f1569810640 1 -- 172.21.15.83:0/1350522599 wait complete. 2024-04-25T11:41:42.228 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:41:42.229 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:41:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.348+0000 7f0c5d689640 1 -- 172.21.15.83:0/4223526098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c58101fa0 msgr2=0x7f0c581023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.348+0000 7f0c5d689640 1 --2- 172.21.15.83:0/4223526098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c58101fa0 0x7f0c581023c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009900 tx=0x7f0c4c02a410 comp rx=0 tx=0).stop 2024-04-25T11:41:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.348+0000 7f0c5d689640 1 -- 172.21.15.83:0/4223526098 shutdown_connections 2024-04-25T11:41:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.348+0000 7f0c5d689640 1 --2- 172.21.15.83:0/4223526098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c58103b50 0x7f0c5810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.348+0000 7f0c5d689640 1 --2- 172.21.15.83:0/4223526098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c58103190 0x7f0c58103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.348+0000 7f0c5d689640 1 --2- 172.21.15.83:0/4223526098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c58101fa0 0x7f0c581023c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.348+0000 7f0c5d689640 1 -- 172.21.15.83:0/4223526098 >> 172.21.15.83:0/4223526098 conn(0x7f0c580fd530 msgr2=0x7f0c580ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.348+0000 7f0c5d689640 1 -- 172.21.15.83:0/4223526098 shutdown_connections 2024-04-25T11:41:42.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.348+0000 7f0c5d689640 1 -- 172.21.15.83:0/4223526098 wait complete. 2024-04-25T11:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.349+0000 7f0c5d689640 1 Processor -- start 2024-04-25T11:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.349+0000 7f0c5d689640 1 -- start start 2024-04-25T11:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.349+0000 7f0c5d689640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c58101fa0 0x7f0c58198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.349+0000 7f0c5d689640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c58103190 0x7f0c581986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.349+0000 7f0c5d689640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c58103b50 0x7f0c5819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.349+0000 7f0c5d689640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5810dae0 con 0x7f0c58101fa0 2024-04-25T11:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.349+0000 7f0c5d689640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5810d960 con 0x7f0c58103b50 2024-04-25T11:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.349+0000 7f0c5d689640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5810dc60 con 0x7f0c58103190 2024-04-25T11:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.349+0000 7f0c56ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c58101fa0 0x7f0c58198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.349+0000 7f0c56ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c58101fa0 0x7f0c58198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44046/0 (socket says 172.21.15.83:44046) 2024-04-25T11:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.349+0000 7f0c56ffd640 1 -- 172.21.15.83:0/1742909573 learned_addr learned my addr 172.21.15.83:0/1742909573 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.350+0000 7f0c567fc640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c58103190 0x7f0c581986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.350+0000 7f0c577fe640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c58103b50 0x7f0c5819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:42.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.350+0000 7f0c56ffd640 1 -- 172.21.15.83:0/1742909573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c58103190 msgr2=0x7f0c581986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.350+0000 7f0c56ffd640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c58103190 0x7f0c581986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.350+0000 7f0c56ffd640 1 -- 172.21.15.83:0/1742909573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c58103b50 msgr2=0x7f0c5819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.350+0000 7f0c56ffd640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c58103b50 0x7f0c5819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.350+0000 7f0c56ffd640 1 -- 172.21.15.83:0/1742909573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5819fda0 con 0x7f0c58101fa0 2024-04-25T11:41:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.350+0000 7f0c577fe640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c58103b50 0x7f0c5819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.350+0000 7f0c567fc640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c58103190 0x7f0c581986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.350+0000 7f0c56ffd640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c58101fa0 0x7f0c58198180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c006010 tx=0x7f0c4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.350+0000 7f0c3ffff640 1 -- 172.21.15.83:0/1742909573 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c03f030 con 0x7f0c58101fa0 2024-04-25T11:41:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.350+0000 7f0c3ffff640 1 -- 172.21.15.83:0/1742909573 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c4c003cb0 con 0x7f0c58101fa0 2024-04-25T11:41:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.350+0000 7f0c5d689640 1 -- 172.21.15.83:0/1742909573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c581a0030 con 0x7f0c58101fa0 2024-04-25T11:41:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.350+0000 7f0c3ffff640 1 -- 172.21.15.83:0/1742909573 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c035070 con 0x7f0c58101fa0 2024-04-25T11:41:42.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.350+0000 7f0c5d689640 1 -- 172.21.15.83:0/1742909573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c581a0520 con 0x7f0c58101fa0 2024-04-25T11:41:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.351+0000 7f0c5d689640 1 -- 172.21.15.83:0/1742909573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c58109390 con 0x7f0c58101fa0 2024-04-25T11:41:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.352+0000 7f0c3ffff640 1 -- 172.21.15.83:0/1742909573 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0c4c042030 con 0x7f0c58101fa0 2024-04-25T11:41:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.353+0000 7f0c3ffff640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0c400a3bb0 0x7f0c400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.353+0000 7f0c567fc640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0c400a3bb0 0x7f0c400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.353+0000 7f0c3ffff640 1 -- 172.21.15.83:0/1742909573 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0c4c0f06f0 con 0x7f0c58101fa0 2024-04-25T11:41:42.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.354+0000 7f0c567fc640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0c400a3bb0 0x7f0c400a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0c58199690 tx=0x7f0c4800a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:42.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.358+0000 7f0c3ffff640 1 -- 172.21.15.83:0/1742909573 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c4c0bd4d0 con 0x7f0c58101fa0 2024-04-25T11:41:42.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:42.542+0000 7f0c5d689640 1 -- 172.21.15.83:0/1742909573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0c5802d5c0 con 0x7f0c58101fa0 2024-04-25T11:41:42.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:42 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/434887448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:42.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:42 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3287668873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:41:42.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:42 smithi111 ceph-mon[32784]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:42.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:42 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:42 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:42 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:42 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:42 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:42 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:42 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:42.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:42 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:41:42.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:42 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:42 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1350522599' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T11:41:42.572 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:42 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:41:42.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:42 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/434887448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:42.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:42 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3287668873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:41:42.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:42 smithi083 ceph-mon[29330]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:42.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:42 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:42 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:42 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:42 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:42 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:42 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:42 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:42.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:42 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:41:42.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:42 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:42 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1350522599' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T11:41:42.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:42 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:41:42.716 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:42 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/434887448' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:42.716 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:42 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3287668873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:41:42.716 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:42 smithi094 ceph-mon[33032]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:42.716 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:42 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.716 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:42 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.716 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:42 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.716 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:42 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.716 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:42 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.716 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:42 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.716 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:42 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:42.716 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:42 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:41:42.717 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:42 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:42.717 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:42 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1350522599' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T11:41:42.717 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:42 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:41:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.429+0000 7f0c3ffff640 1 -- 172.21.15.83:0/1742909573 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f0c4c0bcdd0 con 0x7f0c58101fa0 2024-04-25T11:41:43.417 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-25T11:41:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.431+0000 7f0c5d689640 1 -- 172.21.15.83:0/1742909573 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0c400a3bb0 msgr2=0x7f0c400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.431+0000 7f0c5d689640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0c400a3bb0 0x7f0c400a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0c58199690 tx=0x7f0c4800a5c0 comp rx=0 tx=0).stop 2024-04-25T11:41:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.431+0000 7f0c5d689640 1 -- 172.21.15.83:0/1742909573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c58101fa0 msgr2=0x7f0c58198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.431+0000 7f0c5d689640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c58101fa0 0x7f0c58198180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c006010 tx=0x7f0c4c0042f0 comp rx=0 tx=0).stop 2024-04-25T11:41:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.431+0000 7f0c5d689640 1 -- 172.21.15.83:0/1742909573 shutdown_connections 2024-04-25T11:41:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.431+0000 7f0c5d689640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0c400a3bb0 0x7f0c400a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.431+0000 7f0c5d689640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c58103b50 0x7f0c5819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.431+0000 7f0c5d689640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c58103190 0x7f0c581986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.431+0000 7f0c5d689640 1 --2- 172.21.15.83:0/1742909573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c58101fa0 0x7f0c58198180 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.431+0000 7f0c5d689640 1 -- 172.21.15.83:0/1742909573 >> 172.21.15.83:0/1742909573 conn(0x7f0c580fd530 msgr2=0x7f0c58106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.431+0000 7f0c5d689640 1 -- 172.21.15.83:0/1742909573 shutdown_connections 2024-04-25T11:41:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.431+0000 7f0c5d689640 1 -- 172.21.15.83:0/1742909573 wait complete. 2024-04-25T11:41:43.433 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:41:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.553+0000 7f047f09a640 1 -- 172.21.15.83:0/859913519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04781013c0 msgr2=0x7f04781017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.553+0000 7f047f09a640 1 --2- 172.21.15.83:0/859913519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04781013c0 0x7f04781017e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f046c009900 tx=0x7f046c02a410 comp rx=0 tx=0).stop 2024-04-25T11:41:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.553+0000 7f047f09a640 1 -- 172.21.15.83:0/859913519 shutdown_connections 2024-04-25T11:41:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.553+0000 7f047f09a640 1 --2- 172.21.15.83:0/859913519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0478102a90 0x7f0478109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:43.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.553+0000 7f047f09a640 1 --2- 172.21.15.83:0/859913519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04781020d0 0x7f0478102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.553+0000 7f047f09a640 1 --2- 172.21.15.83:0/859913519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04781013c0 0x7f04781017e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.553+0000 7f047f09a640 1 -- 172.21.15.83:0/859913519 >> 172.21.15.83:0/859913519 conn(0x7f04780fbb90 msgr2=0x7f04780fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.553+0000 7f047f09a640 1 -- 172.21.15.83:0/859913519 shutdown_connections 2024-04-25T11:41:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.553+0000 7f047f09a640 1 -- 172.21.15.83:0/859913519 wait complete. 2024-04-25T11:41:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.554+0000 7f047f09a640 1 Processor -- start 2024-04-25T11:41:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.554+0000 7f047f09a640 1 -- start start 2024-04-25T11:41:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.554+0000 7f047f09a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04781013c0 0x7f0478198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.554+0000 7f047f09a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04781020d0 0x7f0478198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.554+0000 7f047f09a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0478102a90 0x7f047819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.554+0000 7f047f09a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f047810c9c0 con 0x7f04781013c0 2024-04-25T11:41:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.554+0000 7f047f09a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f047810c840 con 0x7f0478102a90 2024-04-25T11:41:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.554+0000 7f047f09a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f047810cb40 con 0x7f04781020d0 2024-04-25T11:41:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.554+0000 7f047ce36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04781013c0 0x7f0478198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:43.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.554+0000 7f047ce36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04781013c0 0x7f0478198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43002/0 (socket says 172.21.15.83:43002) 2024-04-25T11:41:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.554+0000 7f047ce36640 1 -- 172.21.15.83:0/2159048531 learned_addr learned my addr 172.21.15.83:0/2159048531 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.554+0000 7f0477fff640 1 --2- 172.21.15.83:0/2159048531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04781020d0 0x7f0478198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.554+0000 7f047d637640 1 --2- 172.21.15.83:0/2159048531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0478102a90 0x7f047819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.555+0000 7f047ce36640 1 -- 172.21.15.83:0/2159048531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04781020d0 msgr2=0x7f0478198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.555+0000 7f047ce36640 1 --2- 172.21.15.83:0/2159048531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04781020d0 0x7f0478198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.555+0000 7f047ce36640 1 -- 172.21.15.83:0/2159048531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0478102a90 msgr2=0x7f047819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.555+0000 7f047ce36640 1 --2- 172.21.15.83:0/2159048531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0478102a90 0x7f047819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.555+0000 7f047ce36640 1 -- 172.21.15.83:0/2159048531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047819fe30 con 0x7f04781013c0 2024-04-25T11:41:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.555+0000 7f047ce36640 1 --2- 172.21.15.83:0/2159048531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04781013c0 0x7f0478198210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f046c006010 tx=0x7f046c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.555+0000 7f0475ffb640 1 -- 172.21.15.83:0/2159048531 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c03f030 con 0x7f04781013c0 2024-04-25T11:41:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.555+0000 7f047f09a640 1 -- 172.21.15.83:0/2159048531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04781a00c0 con 0x7f04781013c0 2024-04-25T11:41:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.555+0000 7f0475ffb640 1 -- 172.21.15.83:0/2159048531 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f046c0318f0 con 0x7f04781013c0 2024-04-25T11:41:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.555+0000 7f0475ffb640 1 -- 172.21.15.83:0/2159048531 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c035070 con 0x7f04781013c0 2024-04-25T11:41:43.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.555+0000 7f047f09a640 1 -- 172.21.15.83:0/2159048531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04781a05b0 con 0x7f04781013c0 2024-04-25T11:41:43.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.556+0000 7f047f09a640 1 -- 172.21.15.83:0/2159048531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04781a0810 con 0x7f04781013c0 2024-04-25T11:41:43.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.558+0000 7f0475ffb640 1 -- 172.21.15.83:0/2159048531 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f046c042030 con 0x7f04781013c0 2024-04-25T11:41:43.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.558+0000 7f0475ffb640 1 --2- 172.21.15.83:0/2159048531 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f04680a3bb0 0x7f04680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:43.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.558+0000 7f0477fff640 1 --2- 172.21.15.83:0/2159048531 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f04680a3bb0 0x7f04680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:43.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.558+0000 7f0475ffb640 1 -- 172.21.15.83:0/2159048531 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f046c0f0990 con 0x7f04781013c0 2024-04-25T11:41:43.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.559+0000 7f0477fff640 1 --2- 172.21.15.83:0/2159048531 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f04680a3bb0 0x7f04680a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0478199720 tx=0x7f046400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:43.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.562+0000 7f0475ffb640 1 -- 172.21.15.83:0/2159048531 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f046c0bd030 con 0x7f04781013c0 2024-04-25T11:41:43.655 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:43 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1742909573' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T11:41:43.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:43.758+0000 7f047f09a640 1 -- 172.21.15.83:0/2159048531 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f04781a0f20 con 0x7f04781013c0 2024-04-25T11:41:43.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:43 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1742909573' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T11:41:43.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:43 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1742909573' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T11:41:44.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.435+0000 7f0475ffb640 1 -- 172.21.15.83:0/2159048531 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f046c0c1330 con 0x7f04781013c0 2024-04-25T11:41:44.423 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-25T11:41:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.437+0000 7f047f09a640 1 -- 172.21.15.83:0/2159048531 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f04680a3bb0 msgr2=0x7f04680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.437+0000 7f047f09a640 1 --2- 172.21.15.83:0/2159048531 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f04680a3bb0 0x7f04680a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0478199720 tx=0x7f046400b410 comp rx=0 tx=0).stop 2024-04-25T11:41:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.437+0000 7f047f09a640 1 -- 172.21.15.83:0/2159048531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04781013c0 msgr2=0x7f0478198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:44.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.437+0000 7f047f09a640 1 --2- 172.21.15.83:0/2159048531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04781013c0 0x7f0478198210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f046c006010 tx=0x7f046c0042f0 comp rx=0 tx=0).stop 2024-04-25T11:41:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.438+0000 7f047f09a640 1 -- 172.21.15.83:0/2159048531 shutdown_connections 2024-04-25T11:41:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.438+0000 7f047f09a640 1 --2- 172.21.15.83:0/2159048531 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f04680a3bb0 0x7f04680a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.438+0000 7f047f09a640 1 --2- 172.21.15.83:0/2159048531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0478102a90 0x7f047819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.438+0000 7f047f09a640 1 --2- 172.21.15.83:0/2159048531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04781020d0 0x7f0478198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.438+0000 7f047f09a640 1 --2- 172.21.15.83:0/2159048531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04781013c0 0x7f0478198210 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.438+0000 7f047f09a640 1 -- 172.21.15.83:0/2159048531 >> 172.21.15.83:0/2159048531 conn(0x7f04780fbb90 msgr2=0x7f04780fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:44.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.438+0000 7f047f09a640 1 -- 172.21.15.83:0/2159048531 shutdown_connections 2024-04-25T11:41:44.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.438+0000 7f047f09a640 1 -- 172.21.15.83:0/2159048531 wait complete. 2024-04-25T11:41:44.448 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:41:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.561+0000 7fb4ecfad640 1 -- 172.21.15.83:0/2822189194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e8101fa0 msgr2=0x7fb4e81023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:44.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.561+0000 7fb4ecfad640 1 --2- 172.21.15.83:0/2822189194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e8101fa0 0x7fb4e81023c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009900 tx=0x7fb4d802a410 comp rx=0 tx=0).stop 2024-04-25T11:41:44.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.561+0000 7fb4ecfad640 1 -- 172.21.15.83:0/2822189194 shutdown_connections 2024-04-25T11:41:44.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.561+0000 7fb4ecfad640 1 --2- 172.21.15.83:0/2822189194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4e8103b50 0x7fb4e810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:44.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.561+0000 7fb4ecfad640 1 --2- 172.21.15.83:0/2822189194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4e8103190 0x7fb4e8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:44.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.561+0000 7fb4ecfad640 1 --2- 172.21.15.83:0/2822189194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e8101fa0 0x7fb4e81023c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:44.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.561+0000 7fb4ecfad640 1 -- 172.21.15.83:0/2822189194 >> 172.21.15.83:0/2822189194 conn(0x7fb4e80fd530 msgr2=0x7fb4e80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:44.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.561+0000 7fb4ecfad640 1 -- 172.21.15.83:0/2822189194 shutdown_connections 2024-04-25T11:41:44.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.561+0000 7fb4ecfad640 1 -- 172.21.15.83:0/2822189194 wait complete. 2024-04-25T11:41:44.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.562+0000 7fb4ecfad640 1 Processor -- start 2024-04-25T11:41:44.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.562+0000 7fb4ecfad640 1 -- start start 2024-04-25T11:41:44.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.562+0000 7fb4ecfad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4e8101fa0 0x7fb4e8198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:44.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.562+0000 7fb4ecfad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e8103190 0x7fb4e8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.562+0000 7fb4ecfad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4e8103b50 0x7fb4e819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.562+0000 7fb4ecfad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e810dae0 con 0x7fb4e8103190 2024-04-25T11:41:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.562+0000 7fb4ecfad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e810d960 con 0x7fb4e8103b50 2024-04-25T11:41:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.562+0000 7fb4ecfad640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e810dc60 con 0x7fb4e8101fa0 2024-04-25T11:41:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.562+0000 7fb4e5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e8103190 0x7fb4e8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.562+0000 7fb4e5d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e8103190 0x7fb4e8198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43010/0 (socket says 172.21.15.83:43010) 2024-04-25T11:41:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.562+0000 7fb4e5d9b640 1 -- 172.21.15.83:0/1442374390 learned_addr learned my addr 172.21.15.83:0/1442374390 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.562+0000 7fb4e659c640 1 --2- 172.21.15.83:0/1442374390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4e8101fa0 0x7fb4e8198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.563+0000 7fb4e6d9d640 1 --2- 172.21.15.83:0/1442374390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4e8103b50 0x7fb4e819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.563+0000 7fb4e5d9b640 1 -- 172.21.15.83:0/1442374390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4e8101fa0 msgr2=0x7fb4e8198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.563+0000 7fb4e5d9b640 1 --2- 172.21.15.83:0/1442374390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4e8101fa0 0x7fb4e8198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.563+0000 7fb4e5d9b640 1 -- 172.21.15.83:0/1442374390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4e8103b50 msgr2=0x7fb4e819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.563+0000 7fb4e5d9b640 1 --2- 172.21.15.83:0/1442374390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4e8103b50 0x7fb4e819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.563+0000 7fb4e5d9b640 1 -- 172.21.15.83:0/1442374390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e819fe40 con 0x7fb4e8103190 2024-04-25T11:41:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.563+0000 7fb4e6d9d640 1 --2- 172.21.15.83:0/1442374390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4e8103b50 0x7fb4e819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.563+0000 7fb4e5d9b640 1 --2- 172.21.15.83:0/1442374390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e8103190 0x7fb4e8198760 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00d770 tx=0x7fb4dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.563+0000 7fb4d77fe640 1 -- 172.21.15.83:0/1442374390 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc017070 con 0x7fb4e8103190 2024-04-25T11:41:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.563+0000 7fb4d77fe640 1 -- 172.21.15.83:0/1442374390 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4dc0054d0 con 0x7fb4e8103190 2024-04-25T11:41:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.563+0000 7fb4ecfad640 1 -- 172.21.15.83:0/1442374390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e81a0130 con 0x7fb4e8103190 2024-04-25T11:41:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.563+0000 7fb4d77fe640 1 -- 172.21.15.83:0/1442374390 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc004c30 con 0x7fb4e8103190 2024-04-25T11:41:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.563+0000 7fb4ecfad640 1 -- 172.21.15.83:0/1442374390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e81a0630 con 0x7fb4e8103190 2024-04-25T11:41:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.564+0000 7fb4ecfad640 1 -- 172.21.15.83:0/1442374390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e8109430 con 0x7fb4e8103190 2024-04-25T11:41:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.565+0000 7fb4d77fe640 1 -- 172.21.15.83:0/1442374390 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb4dc004e10 con 0x7fb4e8103190 2024-04-25T11:41:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.566+0000 7fb4d77fe640 1 --2- 172.21.15.83:0/1442374390 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb4d00a3bb0 0x7fb4d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.566+0000 7fb4e659c640 1 --2- 172.21.15.83:0/1442374390 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb4d00a3bb0 0x7fb4d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.566+0000 7fb4d77fe640 1 -- 172.21.15.83:0/1442374390 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7fb4dc0cfd10 con 0x7fb4e8103190 2024-04-25T11:41:44.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.566+0000 7fb4e659c640 1 --2- 172.21.15.83:0/1442374390 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb4d00a3bb0 0x7fb4d00a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009230 tx=0x7fb4d80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:44.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.571+0000 7fb4d77fe640 1 -- 172.21.15.83:0/1442374390 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4dc09c8a0 con 0x7fb4e8103190 2024-04-25T11:41:44.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:44 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1742909573' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T11:41:44.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:44 smithi094 ceph-mon[33032]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T11:41:44.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:44 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2159048531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T11:41:44.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:44 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:44 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:44 smithi094 ceph-mon[33032]: pgmap v280: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:44.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:44 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:44 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:44 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:44 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:44 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:44.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:44 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:41:44.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:44 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:44.785+0000 7fb4ecfad640 1 -- 172.21.15.83:0/1442374390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb4e8062bc0 con 0x7fb4e8103190 2024-04-25T11:41:44.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:44 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1742909573' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T11:41:44.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:44 smithi111 ceph-mon[32784]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T11:41:44.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:44 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2159048531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T11:41:44.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:44 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:44 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:44 smithi111 ceph-mon[32784]: pgmap v280: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:44.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:44 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:44 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:44 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:44 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:44 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:44.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:44 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:41:44.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:44 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:44 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1742909573' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T11:41:44.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:44 smithi083 ceph-mon[29330]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T11:41:44.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:44 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2159048531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T11:41:44.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:44 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:44 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:44 smithi083 ceph-mon[29330]: pgmap v280: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:44.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:44 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:44 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:44 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:44 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:44.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:44 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:41:44.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:44 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:41:44.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:44 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:45.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.436+0000 7fb4d77fe640 1 -- 172.21.15.83:0/1442374390 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fb4dc09c1a0 con 0x7fb4e8103190 2024-04-25T11:41:45.424 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T11:41:45.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.439+0000 7fb4ecfad640 1 -- 172.21.15.83:0/1442374390 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb4d00a3bb0 msgr2=0x7fb4d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:45.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.439+0000 7fb4ecfad640 1 --2- 172.21.15.83:0/1442374390 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb4d00a3bb0 0x7fb4d00a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009230 tx=0x7fb4d80314a0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.439+0000 7fb4ecfad640 1 -- 172.21.15.83:0/1442374390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e8103190 msgr2=0x7fb4e8198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:45.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.439+0000 7fb4ecfad640 1 --2- 172.21.15.83:0/1442374390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e8103190 0x7fb4e8198760 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00d770 tx=0x7fb4dc00dc30 comp rx=0 tx=0).stop 2024-04-25T11:41:45.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.439+0000 7fb4ecfad640 1 -- 172.21.15.83:0/1442374390 shutdown_connections 2024-04-25T11:41:45.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.439+0000 7fb4ecfad640 1 --2- 172.21.15.83:0/1442374390 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb4d00a3bb0 0x7fb4d00a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.439+0000 7fb4ecfad640 1 --2- 172.21.15.83:0/1442374390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4e8103190 0x7fb4e8198760 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.439+0000 7fb4ecfad640 1 --2- 172.21.15.83:0/1442374390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4e8103b50 0x7fb4e819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.439+0000 7fb4ecfad640 1 --2- 172.21.15.83:0/1442374390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4e8101fa0 0x7fb4e8198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.439+0000 7fb4ecfad640 1 -- 172.21.15.83:0/1442374390 >> 172.21.15.83:0/1442374390 conn(0x7fb4e80fd530 msgr2=0x7fb4e8106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:45.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.439+0000 7fb4ecfad640 1 -- 172.21.15.83:0/1442374390 shutdown_connections 2024-04-25T11:41:45.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.439+0000 7fb4ecfad640 1 -- 172.21.15.83:0/1442374390 wait complete. 2024-04-25T11:41:45.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:41:45.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.550+0000 7f270766b640 1 -- 172.21.15.83:0/3593902628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2700100810 msgr2=0x7f2700102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.550+0000 7f270766b640 1 --2- 172.21.15.83:0/3593902628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2700100810 0x7f2700102c40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009900 tx=0x7f26fc02a410 comp rx=0 tx=0).stop 2024-04-25T11:41:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.550+0000 7f270766b640 1 -- 172.21.15.83:0/3593902628 shutdown_connections 2024-04-25T11:41:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.550+0000 7f270766b640 1 --2- 172.21.15.83:0/3593902628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2700105b30 0x7f2700105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.550+0000 7f270766b640 1 --2- 172.21.15.83:0/3593902628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2700103180 0x7f27001055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.550+0000 7f270766b640 1 --2- 172.21.15.83:0/3593902628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2700100810 0x7f2700102c40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.550+0000 7f270766b640 1 -- 172.21.15.83:0/3593902628 >> 172.21.15.83:0/3593902628 conn(0x7f27000fc430 msgr2=0x7f27000fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f270766b640 1 -- 172.21.15.83:0/3593902628 shutdown_connections 2024-04-25T11:41:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f270766b640 1 -- 172.21.15.83:0/3593902628 wait complete. 2024-04-25T11:41:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f270766b640 1 Processor -- start 2024-04-25T11:41:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f270766b640 1 -- start start 2024-04-25T11:41:45.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f270766b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2700100810 0x7f270019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f270766b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2700103180 0x7f270019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f270766b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2700105b30 0x7f27001a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f270766b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2700114120 con 0x7f2700100810 2024-04-25T11:41:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f270766b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2700113fa0 con 0x7f2700103180 2024-04-25T11:41:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f270766b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f27001142a0 con 0x7f2700105b30 2024-04-25T11:41:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f2705407640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2700100810 0x7f270019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f2705407640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2700100810 0x7f270019e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43020/0 (socket says 172.21.15.83:43020) 2024-04-25T11:41:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f2705407640 1 -- 172.21.15.83:0/1110637287 learned_addr learned my addr 172.21.15.83:0/1110637287 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f2704c06640 1 --2- 172.21.15.83:0/1110637287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2700103180 0x7f270019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f2705407640 1 -- 172.21.15.83:0/1110637287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2700105b30 msgr2=0x7f27001a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.551+0000 7f2705407640 1 --2- 172.21.15.83:0/1110637287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2700105b30 0x7f27001a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.552+0000 7f2705407640 1 -- 172.21.15.83:0/1110637287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2700103180 msgr2=0x7f270019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.552+0000 7f2705407640 1 --2- 172.21.15.83:0/1110637287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2700103180 0x7f270019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.552+0000 7f2705407640 1 -- 172.21.15.83:0/1110637287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27001a63f0 con 0x7f2700100810 2024-04-25T11:41:45.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.552+0000 7f2704c06640 1 --2- 172.21.15.83:0/1110637287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2700103180 0x7f270019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:45.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.552+0000 7f2705407640 1 --2- 172.21.15.83:0/1110637287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2700100810 0x7f270019e7d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f26fc031500 tx=0x7f26fc0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:45.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.552+0000 7f26f67fc640 1 -- 172.21.15.83:0/1110637287 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc03f030 con 0x7f2700100810 2024-04-25T11:41:45.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.552+0000 7f26f67fc640 1 -- 172.21.15.83:0/1110637287 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc003800 con 0x7f2700100810 2024-04-25T11:41:45.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.552+0000 7f270766b640 1 -- 172.21.15.83:0/1110637287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27001a6680 con 0x7f2700100810 2024-04-25T11:41:45.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.552+0000 7f26f67fc640 1 -- 172.21.15.83:0/1110637287 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc035070 con 0x7f2700100810 2024-04-25T11:41:45.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.552+0000 7f270766b640 1 -- 172.21.15.83:0/1110637287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27001a6c40 con 0x7f2700100810 2024-04-25T11:41:45.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.553+0000 7f270766b640 1 -- 172.21.15.83:0/1110637287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27000fdf20 con 0x7f2700100810 2024-04-25T11:41:45.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.554+0000 7f26f67fc640 1 -- 172.21.15.83:0/1110637287 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f26fc0039b0 con 0x7f2700100810 2024-04-25T11:41:45.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.554+0000 7f26f67fc640 1 --2- 172.21.15.83:0/1110637287 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f26ec0a3c30 0x7f26ec0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:45.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.554+0000 7f2704c06640 1 --2- 172.21.15.83:0/1110637287 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f26ec0a3c30 0x7f26ec0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:45.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.554+0000 7f26f67fc640 1 -- 172.21.15.83:0/1110637287 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f26fc0f0d20 con 0x7f2700100810 2024-04-25T11:41:45.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.555+0000 7f2704c06640 1 --2- 172.21.15.83:0/1110637287 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f26ec0a3c30 0x7f26ec0a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f26f0006fd0 tx=0x7f26f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:45.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.559+0000 7f26f67fc640 1 -- 172.21.15.83:0/1110637287 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26fc0bd850 con 0x7f2700100810 2024-04-25T11:41:45.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.737+0000 7f270766b640 1 -- 172.21.15.83:0/1110637287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2700062bf0 con 0x7f2700100810 2024-04-25T11:41:45.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.739+0000 7f26f67fc640 1 -- 172.21.15.83:0/1110637287 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22160 (secure 0 0 0) 0x7f26fc0bd150 con 0x7f2700100810 2024-04-25T11:41:45.727 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:45.727 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":85,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:41:44.787965+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:41:42.546991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:41:43.764226+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:41:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.741+0000 7f270766b640 1 -- 172.21.15.83:0/1110637287 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f26ec0a3c30 msgr2=0x7f26ec0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.741+0000 7f270766b640 1 --2- 172.21.15.83:0/1110637287 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f26ec0a3c30 0x7f26ec0a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f26f0006fd0 tx=0x7f26f0008040 comp rx=0 tx=0).stop 2024-04-25T11:41:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.741+0000 7f270766b640 1 -- 172.21.15.83:0/1110637287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2700100810 msgr2=0x7f270019e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.741+0000 7f270766b640 1 --2- 172.21.15.83:0/1110637287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2700100810 0x7f270019e7d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f26fc031500 tx=0x7f26fc0043c0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.741+0000 7f270766b640 1 -- 172.21.15.83:0/1110637287 shutdown_connections 2024-04-25T11:41:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.741+0000 7f270766b640 1 --2- 172.21.15.83:0/1110637287 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f26ec0a3c30 0x7f26ec0a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.741+0000 7f270766b640 1 --2- 172.21.15.83:0/1110637287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2700105b30 0x7f27001a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.741+0000 7f270766b640 1 --2- 172.21.15.83:0/1110637287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2700103180 0x7f270019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.741+0000 7f270766b640 1 --2- 172.21.15.83:0/1110637287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2700100810 0x7f270019e7d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.741+0000 7f270766b640 1 -- 172.21.15.83:0/1110637287 >> 172.21.15.83:0/1110637287 conn(0x7f27000fc430 msgr2=0x7f2700103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.741+0000 7f270766b640 1 -- 172.21.15.83:0/1110637287 shutdown_connections 2024-04-25T11:41:45.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.741+0000 7f270766b640 1 -- 172.21.15.83:0/1110637287 wait complete. 2024-04-25T11:41:45.738 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:45 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2159048531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T11:41:45.738 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:45 smithi083 ceph-mon[29330]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T11:41:45.738 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:45 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1442374390' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T11:41:45.739 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:45 smithi083 ceph-mon[29330]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T11:41:45.739 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:45 smithi083 ceph-mon[29330]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T11:41:45.739 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:45 smithi083 ceph-mon[29330]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T11:41:45.739 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:45 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:45.740 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:41:45.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:45 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2159048531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T11:41:45.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:45 smithi094 ceph-mon[33032]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T11:41:45.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:45 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1442374390' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T11:41:45.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:45 smithi094 ceph-mon[33032]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T11:41:45.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:45 smithi094 ceph-mon[33032]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T11:41:45.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:45 smithi094 ceph-mon[33032]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T11:41:45.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:45 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:45.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:45 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2159048531' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T11:41:45.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:45 smithi111 ceph-mon[32784]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T11:41:45.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:45 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1442374390' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T11:41:45.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:45 smithi111 ceph-mon[32784]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T11:41:45.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:45 smithi111 ceph-mon[32784]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T11:41:45.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:45 smithi111 ceph-mon[32784]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T11:41:45.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:45 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:41:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.849+0000 7ffb32bbb640 1 -- 172.21.15.83:0/3604380335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb2c100630 msgr2=0x7ffb2c100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.849+0000 7ffb32bbb640 1 --2- 172.21.15.83:0/3604380335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb2c100630 0x7ffb2c100a50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009900 tx=0x7ffb2002a410 comp rx=0 tx=0).stop 2024-04-25T11:41:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.849+0000 7ffb32bbb640 1 -- 172.21.15.83:0/3604380335 shutdown_connections 2024-04-25T11:41:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.849+0000 7ffb32bbb640 1 --2- 172.21.15.83:0/3604380335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2c1021e0 0x7ffb2c108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.849+0000 7ffb32bbb640 1 --2- 172.21.15.83:0/3604380335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb2c101820 0x7ffb2c101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.849+0000 7ffb32bbb640 1 --2- 172.21.15.83:0/3604380335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb2c100630 0x7ffb2c100a50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.849+0000 7ffb32bbb640 1 -- 172.21.15.83:0/3604380335 >> 172.21.15.83:0/3604380335 conn(0x7ffb2c0fbbc0 msgr2=0x7ffb2c0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.849+0000 7ffb32bbb640 1 -- 172.21.15.83:0/3604380335 shutdown_connections 2024-04-25T11:41:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.849+0000 7ffb32bbb640 1 -- 172.21.15.83:0/3604380335 wait complete. 2024-04-25T11:41:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb32bbb640 1 Processor -- start 2024-04-25T11:41:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb32bbb640 1 -- start start 2024-04-25T11:41:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb32bbb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2c100630 0x7ffb2c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb32bbb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb2c101820 0x7ffb2c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb32bbb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb2c1021e0 0x7ffb2c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb32bbb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c10ca20 con 0x7ffb2c101820 2024-04-25T11:41:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb32bbb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c10c8a0 con 0x7ffb2c100630 2024-04-25T11:41:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb32bbb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c10cba0 con 0x7ffb2c1021e0 2024-04-25T11:41:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb2bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb2c101820 0x7ffb2c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:45.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb2bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb2c101820 0x7ffb2c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43042/0 (socket says 172.21.15.83:43042) 2024-04-25T11:41:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb2bfff640 1 -- 172.21.15.83:0/433700881 learned_addr learned my addr 172.21.15.83:0/433700881 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb30957640 1 --2- 172.21.15.83:0/433700881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2c100630 0x7ffb2c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb2bfff640 1 -- 172.21.15.83:0/433700881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb2c1021e0 msgr2=0x7ffb2c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb2bfff640 1 --2- 172.21.15.83:0/433700881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb2c1021e0 0x7ffb2c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb2bfff640 1 -- 172.21.15.83:0/433700881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2c100630 msgr2=0x7ffb2c198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb2bfff640 1 --2- 172.21.15.83:0/433700881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2c100630 0x7ffb2c198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb2bfff640 1 -- 172.21.15.83:0/433700881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2c19fe80 con 0x7ffb2c101820 2024-04-25T11:41:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.850+0000 7ffb30957640 1 --2- 172.21.15.83:0/433700881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2c100630 0x7ffb2c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.851+0000 7ffb2bfff640 1 --2- 172.21.15.83:0/433700881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb2c101820 0x7ffb2c1987a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800d770 tx=0x7ffb1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.851+0000 7ffb29ffb640 1 -- 172.21.15.83:0/433700881 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb18017070 con 0x7ffb2c101820 2024-04-25T11:41:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.851+0000 7ffb32bbb640 1 -- 172.21.15.83:0/433700881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2c1a0170 con 0x7ffb2c101820 2024-04-25T11:41:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.851+0000 7ffb29ffb640 1 -- 172.21.15.83:0/433700881 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb180054d0 con 0x7ffb2c101820 2024-04-25T11:41:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.851+0000 7ffb29ffb640 1 -- 172.21.15.83:0/433700881 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb18004c30 con 0x7ffb2c101820 2024-04-25T11:41:45.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.851+0000 7ffb32bbb640 1 -- 172.21.15.83:0/433700881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb2c1a0670 con 0x7ffb2c101820 2024-04-25T11:41:45.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.852+0000 7ffb32bbb640 1 -- 172.21.15.83:0/433700881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb2c1a08d0 con 0x7ffb2c101820 2024-04-25T11:41:45.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.853+0000 7ffb29ffb640 1 -- 172.21.15.83:0/433700881 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffb18004e10 con 0x7ffb2c101820 2024-04-25T11:41:45.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.853+0000 7ffb29ffb640 1 --2- 172.21.15.83:0/433700881 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ffb1c0a3bb0 0x7ffb1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:45.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.853+0000 7ffb30957640 1 --2- 172.21.15.83:0/433700881 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ffb1c0a3bb0 0x7ffb1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:45.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.853+0000 7ffb29ffb640 1 -- 172.21.15.83:0/433700881 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ffb180cfe20 con 0x7ffb2c101820 2024-04-25T11:41:45.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.854+0000 7ffb30957640 1 --2- 172.21.15.83:0/433700881 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ffb1c0a3bb0 0x7ffb1c0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffb20000c00 tx=0x7ffb200314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:45.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:45.858+0000 7ffb29ffb640 1 -- 172.21.15.83:0/433700881 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb1809c950 con 0x7ffb2c101820 2024-04-25T11:41:46.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.040+0000 7ffb32bbb640 1 -- 172.21.15.83:0/433700881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb2c1a0bc0 con 0x7ffb2c101820 2024-04-25T11:41:46.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.042+0000 7ffb29ffb640 1 -- 172.21.15.83:0/433700881 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22147 (secure 0 0 0) 0x7ffb1809c250 con 0x7ffb2c101820 2024-04-25T11:41:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:46.030 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:41:45.888504+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:41:42.546991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:41:43.764226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:41:46.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.044+0000 7ffb32bbb640 1 -- 172.21.15.83:0/433700881 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ffb1c0a3bb0 msgr2=0x7ffb1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:46.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.044+0000 7ffb32bbb640 1 --2- 172.21.15.83:0/433700881 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ffb1c0a3bb0 0x7ffb1c0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffb20000c00 tx=0x7ffb200314a0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.044+0000 7ffb32bbb640 1 -- 172.21.15.83:0/433700881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb2c101820 msgr2=0x7ffb2c1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:46.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.044+0000 7ffb32bbb640 1 --2- 172.21.15.83:0/433700881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb2c101820 0x7ffb2c1987a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800d770 tx=0x7ffb1800dc30 comp rx=0 tx=0).stop 2024-04-25T11:41:46.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.044+0000 7ffb32bbb640 1 -- 172.21.15.83:0/433700881 shutdown_connections 2024-04-25T11:41:46.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.044+0000 7ffb32bbb640 1 --2- 172.21.15.83:0/433700881 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ffb1c0a3bb0 0x7ffb1c0a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.044+0000 7ffb32bbb640 1 --2- 172.21.15.83:0/433700881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb2c101820 0x7ffb2c1987a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.044+0000 7ffb32bbb640 1 --2- 172.21.15.83:0/433700881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb2c1021e0 0x7ffb2c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.044+0000 7ffb32bbb640 1 --2- 172.21.15.83:0/433700881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb2c100630 0x7ffb2c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.044+0000 7ffb32bbb640 1 -- 172.21.15.83:0/433700881 >> 172.21.15.83:0/433700881 conn(0x7ffb2c0fbbc0 msgr2=0x7ffb2c1091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:46.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.044+0000 7ffb32bbb640 1 -- 172.21.15.83:0/433700881 shutdown_connections 2024-04-25T11:41:46.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.044+0000 7ffb32bbb640 1 -- 172.21.15.83:0/433700881 wait complete. 2024-04-25T11:41:46.043 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:41:46.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.155+0000 7f7568e33640 1 -- 172.21.15.83:0/2416372870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564100e60 msgr2=0x7f7564101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:46.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.155+0000 7f7568e33640 1 --2- 172.21.15.83:0/2416372870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564100e60 0x7f7564101280 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7554009900 tx=0x7f755402a410 comp rx=0 tx=0).stop 2024-04-25T11:41:46.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.155+0000 7f7568e33640 1 -- 172.21.15.83:0/2416372870 shutdown_connections 2024-04-25T11:41:46.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.155+0000 7f7568e33640 1 --2- 172.21.15.83:0/2416372870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7564102a00 0x7f7564109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.155+0000 7f7568e33640 1 --2- 172.21.15.83:0/2416372870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7564102040 0x7f75641024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.155+0000 7f7568e33640 1 --2- 172.21.15.83:0/2416372870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564100e60 0x7f7564101280 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.155+0000 7f7568e33640 1 -- 172.21.15.83:0/2416372870 >> 172.21.15.83:0/2416372870 conn(0x7f75640fc440 msgr2=0x7f75640fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:46.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.155+0000 7f7568e33640 1 -- 172.21.15.83:0/2416372870 shutdown_connections 2024-04-25T11:41:46.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.155+0000 7f7568e33640 1 -- 172.21.15.83:0/2416372870 wait complete. 2024-04-25T11:41:46.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f7568e33640 1 Processor -- start 2024-04-25T11:41:46.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f7568e33640 1 -- start start 2024-04-25T11:41:46.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f7568e33640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564100e60 0x7f756406b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:46.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f7568e33640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7564102040 0x7f756419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:46.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f7568e33640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7564102a00 0x7f756419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:46.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f7568e33640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f756410cab0 con 0x7f7564100e60 2024-04-25T11:41:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f7568e33640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f756410c930 con 0x7f7564102a00 2024-04-25T11:41:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f7568e33640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f756410cc30 con 0x7f7564102040 2024-04-25T11:41:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f756259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564100e60 0x7f756406b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f756259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564100e60 0x7f756406b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43052/0 (socket says 172.21.15.83:43052) 2024-04-25T11:41:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f756259c640 1 -- 172.21.15.83:0/1434580485 learned_addr learned my addr 172.21.15.83:0/1434580485 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f756259c640 1 -- 172.21.15.83:0/1434580485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7564102040 msgr2=0x7f756419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f756259c640 1 --2- 172.21.15.83:0/1434580485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7564102040 0x7f756419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f756259c640 1 -- 172.21.15.83:0/1434580485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7564102a00 msgr2=0x7f756419f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f756259c640 1 --2- 172.21.15.83:0/1434580485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7564102a00 0x7f756419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f756259c640 1 -- 172.21.15.83:0/1434580485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756419fe80 con 0x7f7564100e60 2024-04-25T11:41:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f756259c640 1 --2- 172.21.15.83:0/1434580485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564100e60 0x7f756406b0e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7554031500 tx=0x7f7554003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f75537fe640 1 -- 172.21.15.83:0/1434580485 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755403f070 con 0x7f7564100e60 2024-04-25T11:41:46.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f7568e33640 1 -- 172.21.15.83:0/1434580485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75641a0110 con 0x7f7564100e60 2024-04-25T11:41:46.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f75537fe640 1 -- 172.21.15.83:0/1434580485 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f755403b460 con 0x7f7564100e60 2024-04-25T11:41:46.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f75537fe640 1 -- 172.21.15.83:0/1434580485 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554035030 con 0x7f7564100e60 2024-04-25T11:41:46.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.156+0000 7f7568e33640 1 -- 172.21.15.83:0/1434580485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75641a06d0 con 0x7f7564100e60 2024-04-25T11:41:46.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.157+0000 7f7568e33640 1 -- 172.21.15.83:0/1434580485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7564108250 con 0x7f7564100e60 2024-04-25T11:41:46.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.158+0000 7f75537fe640 1 -- 172.21.15.83:0/1434580485 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7554042070 con 0x7f7564100e60 2024-04-25T11:41:46.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.159+0000 7f75537fe640 1 --2- 172.21.15.83:0/1434580485 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f754c0a3bb0 0x7f754c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:46.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.159+0000 7f7561d9b640 1 --2- 172.21.15.83:0/1434580485 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f754c0a3bb0 0x7f754c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:46.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.159+0000 7f75537fe640 1 -- 172.21.15.83:0/1434580485 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f75540f0d00 con 0x7f7564100e60 2024-04-25T11:41:46.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.159+0000 7f7561d9b640 1 --2- 172.21.15.83:0/1434580485 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f754c0a3bb0 0x7f754c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f756406ae50 tx=0x7f7558006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.163+0000 7f75537fe640 1 -- 172.21.15.83:0/1434580485 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f75540bd830 con 0x7f7564100e60 2024-04-25T11:41:46.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.380+0000 7f7568e33640 1 -- 172.21.15.83:0/1434580485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f756402d7b0 con 0x7f7564100e60 2024-04-25T11:41:46.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.445+0000 7f75537fe640 1 -- 172.21.15.83:0/1434580485 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f75540bd130 con 0x7f7564100e60 2024-04-25T11:41:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.447+0000 7f7568e33640 1 -- 172.21.15.83:0/1434580485 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f754c0a3bb0 msgr2=0x7f754c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.447+0000 7f7568e33640 1 --2- 172.21.15.83:0/1434580485 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f754c0a3bb0 0x7f754c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f756406ae50 tx=0x7f7558006c90 comp rx=0 tx=0).stop 2024-04-25T11:41:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.447+0000 7f7568e33640 1 -- 172.21.15.83:0/1434580485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564100e60 msgr2=0x7f756406b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.447+0000 7f7568e33640 1 --2- 172.21.15.83:0/1434580485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564100e60 0x7f756406b0e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7554031500 tx=0x7f7554003d50 comp rx=0 tx=0).stop 2024-04-25T11:41:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.447+0000 7f7568e33640 1 -- 172.21.15.83:0/1434580485 shutdown_connections 2024-04-25T11:41:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.447+0000 7f7568e33640 1 --2- 172.21.15.83:0/1434580485 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f754c0a3bb0 0x7f754c0a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.447+0000 7f7568e33640 1 --2- 172.21.15.83:0/1434580485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7564102a00 0x7f756419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.447+0000 7f7568e33640 1 --2- 172.21.15.83:0/1434580485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7564102040 0x7f756419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.447+0000 7f7568e33640 1 --2- 172.21.15.83:0/1434580485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7564100e60 0x7f756406b0e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.447+0000 7f7568e33640 1 -- 172.21.15.83:0/1434580485 >> 172.21.15.83:0/1434580485 conn(0x7f75640fc440 msgr2=0x7f7564105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.447+0000 7f7568e33640 1 -- 172.21.15.83:0/1434580485 shutdown_connections 2024-04-25T11:41:46.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.447+0000 7f7568e33640 1 -- 172.21.15.83:0/1434580485 wait complete. 2024-04-25T11:41:46.447 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-25T11:41:46.447 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-25T11:41:46.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.560+0000 7f4787a05640 1 -- 172.21.15.83:0/1718496742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4780100e90 msgr2=0x7f47801012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:46.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.560+0000 7f4787a05640 1 --2- 172.21.15.83:0/1718496742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4780100e90 0x7f47801012b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f477c009900 tx=0x7f477c02a410 comp rx=0 tx=0).stop 2024-04-25T11:41:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.560+0000 7f4787a05640 1 -- 172.21.15.83:0/1718496742 shutdown_connections 2024-04-25T11:41:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.560+0000 7f4787a05640 1 --2- 172.21.15.83:0/1718496742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47801029a0 0x7f47801092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.560+0000 7f4787a05640 1 --2- 172.21.15.83:0/1718496742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4780101fe0 0x7f4780102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.560+0000 7f4787a05640 1 --2- 172.21.15.83:0/1718496742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4780100e90 0x7f47801012b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.560+0000 7f4787a05640 1 -- 172.21.15.83:0/1718496742 >> 172.21.15.83:0/1718496742 conn(0x7f47800fc400 msgr2=0x7f47800fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.561+0000 7f4787a05640 1 -- 172.21.15.83:0/1718496742 shutdown_connections 2024-04-25T11:41:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.561+0000 7f4787a05640 1 -- 172.21.15.83:0/1718496742 wait complete. 2024-04-25T11:41:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.561+0000 7f4787a05640 1 Processor -- start 2024-04-25T11:41:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.561+0000 7f4787a05640 1 -- start start 2024-04-25T11:41:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.561+0000 7f4787a05640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4780100e90 0x7f4780198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.561+0000 7f4787a05640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4780101fe0 0x7f4780198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.561+0000 7f4787a05640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47801029a0 0x7f478019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.561+0000 7f4787a05640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f478010c9c0 con 0x7f4780101fe0 2024-04-25T11:41:46.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.561+0000 7f4787a05640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f478010c840 con 0x7f4780100e90 2024-04-25T11:41:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.561+0000 7f4787a05640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f478010cb40 con 0x7f47801029a0 2024-04-25T11:41:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.561+0000 7f4784fa0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4780101fe0 0x7f4780198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.561+0000 7f4784fa0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4780101fe0 0x7f4780198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43066/0 (socket says 172.21.15.83:43066) 2024-04-25T11:41:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.561+0000 7f4784fa0640 1 -- 172.21.15.83:0/3976174164 learned_addr learned my addr 172.21.15.83:0/3976174164 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.561+0000 7f47857a1640 1 --2- 172.21.15.83:0/3976174164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4780100e90 0x7f4780198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.562+0000 7f4784fa0640 1 -- 172.21.15.83:0/3976174164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47801029a0 msgr2=0x7f478019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.562+0000 7f4784fa0640 1 --2- 172.21.15.83:0/3976174164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47801029a0 0x7f478019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.562+0000 7f4784fa0640 1 -- 172.21.15.83:0/3976174164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4780100e90 msgr2=0x7f4780198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.562+0000 7f4784fa0640 1 --2- 172.21.15.83:0/3976174164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4780100e90 0x7f4780198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.562+0000 7f4784fa0640 1 -- 172.21.15.83:0/3976174164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478019fe30 con 0x7f4780101fe0 2024-04-25T11:41:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.562+0000 7f4784fa0640 1 --2- 172.21.15.83:0/3976174164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4780101fe0 0x7f4780198750 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f477000d770 tx=0x7f477000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.562+0000 7f47767fc640 1 -- 172.21.15.83:0/3976174164 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770017070 con 0x7f4780101fe0 2024-04-25T11:41:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.562+0000 7f47767fc640 1 -- 172.21.15.83:0/3976174164 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4770009c30 con 0x7f4780101fe0 2024-04-25T11:41:46.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.562+0000 7f47767fc640 1 -- 172.21.15.83:0/3976174164 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770003cc0 con 0x7f4780101fe0 2024-04-25T11:41:46.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.562+0000 7f4787a05640 1 -- 172.21.15.83:0/3976174164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47801a0120 con 0x7f4780101fe0 2024-04-25T11:41:46.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.562+0000 7f4787a05640 1 -- 172.21.15.83:0/3976174164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47801a06c0 con 0x7f4780101fe0 2024-04-25T11:41:46.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.563+0000 7f4787a05640 1 -- 172.21.15.83:0/3976174164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47801a0920 con 0x7f4780101fe0 2024-04-25T11:41:46.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.564+0000 7f47767fc640 1 -- 172.21.15.83:0/3976174164 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4770003e60 con 0x7f4780101fe0 2024-04-25T11:41:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.564+0000 7f47767fc640 1 --2- 172.21.15.83:0/3976174164 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f476c0a3c30 0x7f476c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.565+0000 7f47857a1640 1 --2- 172.21.15.83:0/3976174164 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f476c0a3c30 0x7f476c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.565+0000 7f47767fc640 1 -- 172.21.15.83:0/3976174164 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f47700cffd0 con 0x7f4780101fe0 2024-04-25T11:41:46.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.565+0000 7f47857a1640 1 --2- 172.21.15.83:0/3976174164 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f476c0a3c30 0x7f476c0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f477c000c00 tx=0x7f477c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:46.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.568+0000 7f47767fc640 1 -- 172.21.15.83:0/3976174164 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4770009de0 con 0x7f4780101fe0 2024-04-25T11:41:46.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:46.785+0000 7f4787a05640 1 -- 172.21.15.83:0/3976174164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f47801a0cb0 con 0x7f4780101fe0 2024-04-25T11:41:46.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:46 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1442374390' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T11:41:46.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:46 smithi094 ceph-mon[33032]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T11:41:46.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:46 smithi094 ceph-mon[33032]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T11:41:46.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:46 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1110637287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:46.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:46 smithi094 ceph-mon[33032]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T11:41:46.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:46 smithi094 ceph-mon[33032]: pgmap v284: 97 pgs: 17 active+clean, 22 creating+peering, 58 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:46.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:46 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/433700881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:46.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:46 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1434580485' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T11:41:46.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:46 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1442374390' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T11:41:46.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:46 smithi111 ceph-mon[32784]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T11:41:46.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:46 smithi111 ceph-mon[32784]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T11:41:46.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:46 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1110637287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:46.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:46 smithi111 ceph-mon[32784]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T11:41:46.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:46 smithi111 ceph-mon[32784]: pgmap v284: 97 pgs: 17 active+clean, 22 creating+peering, 58 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:46.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:46 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/433700881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:46.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:46 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1434580485' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T11:41:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:46 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1442374390' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T11:41:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:46 smithi083 ceph-mon[29330]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T11:41:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:46 smithi083 ceph-mon[29330]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T11:41:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:46 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1110637287' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:46 smithi083 ceph-mon[29330]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T11:41:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:46 smithi083 ceph-mon[29330]: pgmap v284: 97 pgs: 17 active+clean, 22 creating+peering, 58 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:41:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:46 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/433700881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:46.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:46 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1434580485' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T11:41:47.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.582+0000 7f47767fc640 1 -- 172.21.15.83:0/3976174164 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f477009cb80 con 0x7f4780101fe0 2024-04-25T11:41:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.584+0000 7f4787a05640 1 -- 172.21.15.83:0/3976174164 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f476c0a3c30 msgr2=0x7f476c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.584+0000 7f4787a05640 1 --2- 172.21.15.83:0/3976174164 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f476c0a3c30 0x7f476c0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f477c000c00 tx=0x7f477c0314a0 comp rx=0 tx=0).stop 2024-04-25T11:41:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.584+0000 7f4787a05640 1 -- 172.21.15.83:0/3976174164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4780101fe0 msgr2=0x7f4780198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.584+0000 7f4787a05640 1 --2- 172.21.15.83:0/3976174164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4780101fe0 0x7f4780198750 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f477000d770 tx=0x7f477000dc30 comp rx=0 tx=0).stop 2024-04-25T11:41:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.584+0000 7f4787a05640 1 -- 172.21.15.83:0/3976174164 shutdown_connections 2024-04-25T11:41:47.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.584+0000 7f4787a05640 1 --2- 172.21.15.83:0/3976174164 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f476c0a3c30 0x7f476c0a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.584+0000 7f4787a05640 1 --2- 172.21.15.83:0/3976174164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4780101fe0 0x7f4780198750 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.584+0000 7f4787a05640 1 --2- 172.21.15.83:0/3976174164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47801029a0 0x7f478019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.584+0000 7f4787a05640 1 --2- 172.21.15.83:0/3976174164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4780100e90 0x7f4780198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.584+0000 7f4787a05640 1 -- 172.21.15.83:0/3976174164 >> 172.21.15.83:0/3976174164 conn(0x7f47800fc400 msgr2=0x7f47800690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.585+0000 7f4787a05640 1 -- 172.21.15.83:0/3976174164 shutdown_connections 2024-04-25T11:41:47.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.585+0000 7f4787a05640 1 -- 172.21.15.83:0/3976174164 wait complete. 2024-04-25T11:41:47.585 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T11:41:47.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.699+0000 7f864d0cf640 1 -- 172.21.15.83:0/4292331513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86481021d0 msgr2=0x7f8648108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.699+0000 7f864d0cf640 1 --2- 172.21.15.83:0/4292331513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86481021d0 0x7f8648108a60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8640009ca0 tx=0x7f864002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:41:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.699+0000 7f864d0cf640 1 -- 172.21.15.83:0/4292331513 shutdown_connections 2024-04-25T11:41:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.699+0000 7f864d0cf640 1 --2- 172.21.15.83:0/4292331513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86481021d0 0x7f8648108a60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.699+0000 7f864d0cf640 1 --2- 172.21.15.83:0/4292331513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8648101810 0x7f8648101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.699+0000 7f864d0cf640 1 --2- 172.21.15.83:0/4292331513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648100620 0x7f8648100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.699+0000 7f864d0cf640 1 -- 172.21.15.83:0/4292331513 >> 172.21.15.83:0/4292331513 conn(0x7f86480fbb90 msgr2=0x7f86480fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.699+0000 7f864d0cf640 1 -- 172.21.15.83:0/4292331513 shutdown_connections 2024-04-25T11:41:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.699+0000 7f864d0cf640 1 -- 172.21.15.83:0/4292331513 wait complete. 2024-04-25T11:41:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.699+0000 7f864d0cf640 1 Processor -- start 2024-04-25T11:41:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.699+0000 7f864d0cf640 1 -- start start 2024-04-25T11:41:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864d0cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648100620 0x7f8648197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:47.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864d0cf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8648101810 0x7f8648197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864d0cf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86481021d0 0x7f864819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864d0cf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f864810cac0 con 0x7f86481021d0 2024-04-25T11:41:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864d0cf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f864810c940 con 0x7f8648100620 2024-04-25T11:41:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864d0cf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f864810cc40 con 0x7f8648101810 2024-04-25T11:41:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86481021d0 0x7f864819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86481021d0 0x7f864819e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43072/0 (socket says 172.21.15.83:43072) 2024-04-25T11:41:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864759e640 1 -- 172.21.15.83:0/1104192340 learned_addr learned my addr 172.21.15.83:0/1104192340 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f863ffff640 1 --2- 172.21.15.83:0/1104192340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8648101810 0x7f8648197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f8646d9d640 1 --2- 172.21.15.83:0/1104192340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648100620 0x7f8648197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864759e640 1 -- 172.21.15.83:0/1104192340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8648101810 msgr2=0x7f8648197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864759e640 1 --2- 172.21.15.83:0/1104192340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8648101810 0x7f8648197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864759e640 1 -- 172.21.15.83:0/1104192340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648100620 msgr2=0x7f8648197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:47.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864759e640 1 --2- 172.21.15.83:0/1104192340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648100620 0x7f8648197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:47.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864759e640 1 -- 172.21.15.83:0/1104192340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864819ee20 con 0x7f86481021d0 2024-04-25T11:41:47.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864759e640 1 --2- 172.21.15.83:0/1104192340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86481021d0 0x7f864819e7b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8640031a20 tx=0x7f8640003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:47.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f8644d99640 1 -- 172.21.15.83:0/1104192340 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640004240 con 0x7f86481021d0 2024-04-25T11:41:47.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f8644d99640 1 -- 172.21.15.83:0/1104192340 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8640003e10 con 0x7f86481021d0 2024-04-25T11:41:47.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f8644d99640 1 -- 172.21.15.83:0/1104192340 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864002fd90 con 0x7f86481021d0 2024-04-25T11:41:47.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864d0cf640 1 -- 172.21.15.83:0/1104192340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864819f0b0 con 0x7f86481021d0 2024-04-25T11:41:47.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.700+0000 7f864d0cf640 1 -- 172.21.15.83:0/1104192340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f864819f680 con 0x7f86481021d0 2024-04-25T11:41:47.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.701+0000 7f864d0cf640 1 -- 172.21.15.83:0/1104192340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8648107a60 con 0x7f86481021d0 2024-04-25T11:41:47.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.702+0000 7f8644d99640 1 -- 172.21.15.83:0/1104192340 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8640004b40 con 0x7f86481021d0 2024-04-25T11:41:47.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.703+0000 7f8644d99640 1 --2- 172.21.15.83:0/1104192340 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f86340a3bb0 0x7f86340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:47.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.703+0000 7f8644d99640 1 -- 172.21.15.83:0/1104192340 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f86400f1cf0 con 0x7f86481021d0 2024-04-25T11:41:47.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.703+0000 7f8646d9d640 1 --2- 172.21.15.83:0/1104192340 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f86340a3bb0 0x7f86340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:47.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.703+0000 7f8646d9d640 1 --2- 172.21.15.83:0/1104192340 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f86340a3bb0 0x7f86340a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8648101670 tx=0x7f8638008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:47.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.707+0000 7f8644d99640 1 -- 172.21.15.83:0/1104192340 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f86400be820 con 0x7f86481021d0 2024-04-25T11:41:47.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:47 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1434580485' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T11:41:47.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:47 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] up:active 2024-04-25T11:41:47.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:47 smithi094 ceph-mon[33032]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:41:47.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:47 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3976174164' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T11:41:47.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:47 smithi094 ceph-mon[33032]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T11:41:47.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:47 smithi094 ceph-mon[33032]: Cluster is now healthy 2024-04-25T11:41:47.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:47 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1434580485' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T11:41:47.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:47 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] up:active 2024-04-25T11:41:47.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:47 smithi111 ceph-mon[32784]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:41:47.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:47 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3976174164' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T11:41:47.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:47 smithi111 ceph-mon[32784]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T11:41:47.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:47 smithi111 ceph-mon[32784]: Cluster is now healthy 2024-04-25T11:41:47.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:47 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1434580485' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T11:41:47.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:47 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] up:active 2024-04-25T11:41:47.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:47 smithi083 ceph-mon[29330]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:41:47.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:47 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3976174164' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-25T11:41:47.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:47 smithi083 ceph-mon[29330]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T11:41:47.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:47 smithi083 ceph-mon[29330]: Cluster is now healthy 2024-04-25T11:41:47.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:47.922+0000 7f864d0cf640 1 -- 172.21.15.83:0/1104192340 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f864802d5c0 con 0x7f86481021d0 2024-04-25T11:41:48.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.593+0000 7f8644d99640 1 -- 172.21.15.83:0/1104192340 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7f86400be120 con 0x7f86481021d0 2024-04-25T11:41:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.596+0000 7f864d0cf640 1 -- 172.21.15.83:0/1104192340 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f86340a3bb0 msgr2=0x7f86340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.596+0000 7f864d0cf640 1 --2- 172.21.15.83:0/1104192340 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f86340a3bb0 0x7f86340a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8648101670 tx=0x7f8638008040 comp rx=0 tx=0).stop 2024-04-25T11:41:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.596+0000 7f864d0cf640 1 -- 172.21.15.83:0/1104192340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86481021d0 msgr2=0x7f864819e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.596+0000 7f864d0cf640 1 --2- 172.21.15.83:0/1104192340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86481021d0 0x7f864819e7b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8640031a20 tx=0x7f8640003c20 comp rx=0 tx=0).stop 2024-04-25T11:41:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.596+0000 7f864d0cf640 1 -- 172.21.15.83:0/1104192340 shutdown_connections 2024-04-25T11:41:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.596+0000 7f864d0cf640 1 --2- 172.21.15.83:0/1104192340 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f86340a3bb0 0x7f86340a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.596+0000 7f864d0cf640 1 --2- 172.21.15.83:0/1104192340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86481021d0 0x7f864819e7b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.596+0000 7f864d0cf640 1 --2- 172.21.15.83:0/1104192340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8648101810 0x7f8648197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.596+0000 7f864d0cf640 1 --2- 172.21.15.83:0/1104192340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8648100620 0x7f8648197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.596+0000 7f864d0cf640 1 -- 172.21.15.83:0/1104192340 >> 172.21.15.83:0/1104192340 conn(0x7f86480fbb90 msgr2=0x7f8648104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:48.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.596+0000 7f864d0cf640 1 -- 172.21.15.83:0/1104192340 shutdown_connections 2024-04-25T11:41:48.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.596+0000 7f864d0cf640 1 -- 172.21.15.83:0/1104192340 wait complete. 2024-04-25T11:41:48.600 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T11:41:48.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.717+0000 7f43eee8a640 1 -- 172.21.15.83:0/2179087251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43e8102a70 msgr2=0x7f43e8104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:48.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.717+0000 7f43eee8a640 1 --2- 172.21.15.83:0/2179087251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43e8102a70 0x7f43e8104e60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009900 tx=0x7f43dc02a410 comp rx=0 tx=0).stop 2024-04-25T11:41:48.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.717+0000 7f43eee8a640 1 -- 172.21.15.83:0/2179087251 shutdown_connections 2024-04-25T11:41:48.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.717+0000 7f43eee8a640 1 --2- 172.21.15.83:0/2179087251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43e8107cd0 0x7f43e810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.717+0000 7f43eee8a640 1 --2- 172.21.15.83:0/2179087251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43e81053a0 0x7f43e8107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.717+0000 7f43eee8a640 1 --2- 172.21.15.83:0/2179087251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43e8102a70 0x7f43e8104e60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.717+0000 7f43eee8a640 1 -- 172.21.15.83:0/2179087251 >> 172.21.15.83:0/2179087251 conn(0x7f43e80fc420 msgr2=0x7f43e80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.717+0000 7f43eee8a640 1 -- 172.21.15.83:0/2179087251 shutdown_connections 2024-04-25T11:41:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.717+0000 7f43eee8a640 1 -- 172.21.15.83:0/2179087251 wait complete. 2024-04-25T11:41:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43eee8a640 1 Processor -- start 2024-04-25T11:41:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43eee8a640 1 -- start start 2024-04-25T11:41:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43eee8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43e8102a70 0x7f43e81982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43eee8a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43e81053a0 0x7f43e8198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43eee8a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43e8107cd0 0x7f43e819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43eee8a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43e810db50 con 0x7f43e8102a70 2024-04-25T11:41:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43eee8a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f43e810d9d0 con 0x7f43e81053a0 2024-04-25T11:41:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43eee8a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f43e810dcd0 con 0x7f43e8107cd0 2024-04-25T11:41:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43ecc26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43e8102a70 0x7f43e81982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43ecc26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43e8102a70 0x7f43e81982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43100/0 (socket says 172.21.15.83:43100) 2024-04-25T11:41:48.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43ecc26640 1 -- 172.21.15.83:0/104935518 learned_addr learned my addr 172.21.15.83:0/104935518 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43e7fff640 1 --2- 172.21.15.83:0/104935518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43e81053a0 0x7f43e8198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43ed427640 1 --2- 172.21.15.83:0/104935518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43e8107cd0 0x7f43e819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43ecc26640 1 -- 172.21.15.83:0/104935518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43e8107cd0 msgr2=0x7f43e819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43ecc26640 1 --2- 172.21.15.83:0/104935518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43e8107cd0 0x7f43e819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43ecc26640 1 -- 172.21.15.83:0/104935518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43e81053a0 msgr2=0x7f43e8198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43ecc26640 1 --2- 172.21.15.83:0/104935518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43e81053a0 0x7f43e8198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43ecc26640 1 -- 172.21.15.83:0/104935518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e819fee0 con 0x7f43e8102a70 2024-04-25T11:41:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43ed427640 1 --2- 172.21.15.83:0/104935518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43e8107cd0 0x7f43e819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43ecc26640 1 --2- 172.21.15.83:0/104935518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43e8102a70 0x7f43e81982c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f43dc005940 tx=0x7f43dc004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43e5ffb640 1 -- 172.21.15.83:0/104935518 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc03f030 con 0x7f43e8102a70 2024-04-25T11:41:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43e5ffb640 1 -- 172.21.15.83:0/104935518 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43dc003cb0 con 0x7f43e8102a70 2024-04-25T11:41:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43eee8a640 1 -- 172.21.15.83:0/104935518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e81a0170 con 0x7f43e8102a70 2024-04-25T11:41:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.718+0000 7f43e5ffb640 1 -- 172.21.15.83:0/104935518 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc035070 con 0x7f43e8102a70 2024-04-25T11:41:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.719+0000 7f43eee8a640 1 -- 172.21.15.83:0/104935518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e81a0660 con 0x7f43e8102a70 2024-04-25T11:41:48.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.720+0000 7f43eee8a640 1 -- 172.21.15.83:0/104935518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e80fdf10 con 0x7f43e8102a70 2024-04-25T11:41:48.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.721+0000 7f43e5ffb640 1 -- 172.21.15.83:0/104935518 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f43dc042030 con 0x7f43e8102a70 2024-04-25T11:41:48.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.721+0000 7f43e5ffb640 1 --2- 172.21.15.83:0/104935518 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f43d80a3bb0 0x7f43d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:48.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.721+0000 7f43e7fff640 1 --2- 172.21.15.83:0/104935518 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f43d80a3bb0 0x7f43d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:48.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.721+0000 7f43e5ffb640 1 -- 172.21.15.83:0/104935518 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f43dc0f0860 con 0x7f43e8102a70 2024-04-25T11:41:48.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.722+0000 7f43e7fff640 1 --2- 172.21.15.83:0/104935518 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f43d80a3bb0 0x7f43d80a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f43e81997d0 tx=0x7f43d4009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:48.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.726+0000 7f43e5ffb640 1 -- 172.21.15.83:0/104935518 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43dc0bd390 con 0x7f43e8102a70 2024-04-25T11:41:48.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:48 smithi094 ceph-mon[33032]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T11:41:48.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:48 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3976174164' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T11:41:48.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:48 smithi094 ceph-mon[33032]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T11:41:48.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:48 smithi094 ceph-mon[33032]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:41:48.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:48 smithi094 ceph-mon[33032]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T11:41:48.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:48 smithi094 ceph-mon[33032]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T11:41:48.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:48 smithi094 ceph-mon[33032]: pgmap v285: 97 pgs: 34 active+clean, 22 creating+peering, 41 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 0 op/s 2024-04-25T11:41:48.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:48 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1104192340' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T11:41:48.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:48 smithi111 ceph-mon[32784]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T11:41:48.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:48 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3976174164' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T11:41:48.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:48 smithi111 ceph-mon[32784]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T11:41:48.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:48 smithi111 ceph-mon[32784]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:41:48.821 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:48 smithi111 ceph-mon[32784]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T11:41:48.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:48 smithi111 ceph-mon[32784]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T11:41:48.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:48 smithi111 ceph-mon[32784]: pgmap v285: 97 pgs: 34 active+clean, 22 creating+peering, 41 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 0 op/s 2024-04-25T11:41:48.822 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:48 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1104192340' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T11:41:48.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:48 smithi083 ceph-mon[29330]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T11:41:48.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:48 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3976174164' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-25T11:41:48.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:48 smithi083 ceph-mon[29330]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T11:41:48.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:48 smithi083 ceph-mon[29330]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T11:41:48.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:48 smithi083 ceph-mon[29330]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T11:41:48.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:48 smithi083 ceph-mon[29330]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T11:41:48.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:48 smithi083 ceph-mon[29330]: pgmap v285: 97 pgs: 34 active+clean, 22 creating+peering, 41 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 0 op/s 2024-04-25T11:41:48.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:48 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1104192340' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T11:41:48.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:48.944+0000 7f43eee8a640 1 -- 172.21.15.83:0/104935518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f43e802d5c0 con 0x7f43e8102a70 2024-04-25T11:41:49.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.601+0000 7f43e5ffb640 1 -- 172.21.15.83:0/104935518 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f43dc0bcc90 con 0x7f43e8102a70 2024-04-25T11:41:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.604+0000 7f43eee8a640 1 -- 172.21.15.83:0/104935518 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f43d80a3bb0 msgr2=0x7f43d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.604+0000 7f43eee8a640 1 --2- 172.21.15.83:0/104935518 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f43d80a3bb0 0x7f43d80a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f43e81997d0 tx=0x7f43d4009110 comp rx=0 tx=0).stop 2024-04-25T11:41:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.604+0000 7f43eee8a640 1 -- 172.21.15.83:0/104935518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43e8102a70 msgr2=0x7f43e81982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.604+0000 7f43eee8a640 1 --2- 172.21.15.83:0/104935518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43e8102a70 0x7f43e81982c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f43dc005940 tx=0x7f43dc004430 comp rx=0 tx=0).stop 2024-04-25T11:41:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.604+0000 7f43eee8a640 1 -- 172.21.15.83:0/104935518 shutdown_connections 2024-04-25T11:41:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.604+0000 7f43eee8a640 1 --2- 172.21.15.83:0/104935518 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f43d80a3bb0 0x7f43d80a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.604+0000 7f43eee8a640 1 --2- 172.21.15.83:0/104935518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43e8107cd0 0x7f43e819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.604+0000 7f43eee8a640 1 --2- 172.21.15.83:0/104935518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43e81053a0 0x7f43e8198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.604+0000 7f43eee8a640 1 --2- 172.21.15.83:0/104935518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43e8102a70 0x7f43e81982c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.604+0000 7f43eee8a640 1 -- 172.21.15.83:0/104935518 >> 172.21.15.83:0/104935518 conn(0x7f43e80fc420 msgr2=0x7f43e8103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.604+0000 7f43eee8a640 1 -- 172.21.15.83:0/104935518 shutdown_connections 2024-04-25T11:41:49.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.604+0000 7f43eee8a640 1 -- 172.21.15.83:0/104935518 wait complete. 2024-04-25T11:41:49.608 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-25T11:41:49.608 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T11:41:49.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.726+0000 7f939c3e2640 1 -- 172.21.15.83:0/2700013343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9394100260 msgr2=0x7f9394100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.726+0000 7f939c3e2640 1 --2- 172.21.15.83:0/2700013343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9394100260 0x7f9394100680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9384009930 tx=0x7f938402a420 comp rx=0 tx=0).stop 2024-04-25T11:41:49.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.726+0000 7f939c3e2640 1 -- 172.21.15.83:0/2700013343 shutdown_connections 2024-04-25T11:41:49.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.726+0000 7f939c3e2640 1 --2- 172.21.15.83:0/2700013343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9394103bf0 0x7f939410a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.726+0000 7f939c3e2640 1 --2- 172.21.15.83:0/2700013343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9394103230 0x7f93941036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.726+0000 7f939c3e2640 1 --2- 172.21.15.83:0/2700013343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9394100260 0x7f9394100680 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.726+0000 7f939c3e2640 1 -- 172.21.15.83:0/2700013343 >> 172.21.15.83:0/2700013343 conn(0x7f93940fbb90 msgr2=0x7f93940fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:49.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.726+0000 7f939c3e2640 1 -- 172.21.15.83:0/2700013343 shutdown_connections 2024-04-25T11:41:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.726+0000 7f939c3e2640 1 -- 172.21.15.83:0/2700013343 wait complete. 2024-04-25T11:41:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.726+0000 7f939c3e2640 1 Processor -- start 2024-04-25T11:41:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939c3e2640 1 -- start start 2024-04-25T11:41:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939c3e2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9394100260 0x7f93941982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939c3e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9394103230 0x7f9394198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939c3e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9394103bf0 0x7f939419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939c3e2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f939410db40 con 0x7f9394103230 2024-04-25T11:41:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939c3e2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f939410d9c0 con 0x7f9394103bf0 2024-04-25T11:41:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939c3e2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f939410dcc0 con 0x7f9394100260 2024-04-25T11:41:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939997d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9394103230 0x7f9394198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939997d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9394103230 0x7f9394198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43120/0 (socket says 172.21.15.83:43120) 2024-04-25T11:41:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939997d640 1 -- 172.21.15.83:0/714894811 learned_addr learned my addr 172.21.15.83:0/714894811 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:49.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939a17e640 1 --2- 172.21.15.83:0/714894811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9394100260 0x7f93941982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939a97f640 1 --2- 172.21.15.83:0/714894811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9394103bf0 0x7f939419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939997d640 1 -- 172.21.15.83:0/714894811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9394100260 msgr2=0x7f93941982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939997d640 1 --2- 172.21.15.83:0/714894811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9394100260 0x7f93941982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939997d640 1 -- 172.21.15.83:0/714894811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9394103bf0 msgr2=0x7f939419f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939997d640 1 --2- 172.21.15.83:0/714894811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9394103bf0 0x7f939419f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939997d640 1 -- 172.21.15.83:0/714894811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939419fef0 con 0x7f9394103230 2024-04-25T11:41:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939997d640 1 --2- 172.21.15.83:0/714894811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9394103230 0x7f9394198810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f939000b4a0 tx=0x7f939000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f938b7fe640 1 -- 172.21.15.83:0/714894811 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390014070 con 0x7f9394103230 2024-04-25T11:41:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f938b7fe640 1 -- 172.21.15.83:0/714894811 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9390010ac0 con 0x7f9394103230 2024-04-25T11:41:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939c3e2640 1 -- 172.21.15.83:0/714894811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93941a01e0 con 0x7f9394103230 2024-04-25T11:41:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f938b7fe640 1 -- 172.21.15.83:0/714894811 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390017c30 con 0x7f9394103230 2024-04-25T11:41:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.727+0000 7f939c3e2640 1 -- 172.21.15.83:0/714894811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93941a0800 con 0x7f9394103230 2024-04-25T11:41:49.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.728+0000 7f939c3e2640 1 -- 172.21.15.83:0/714894811 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9394109480 con 0x7f9394103230 2024-04-25T11:41:49.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.730+0000 7f938b7fe640 1 -- 172.21.15.83:0/714894811 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9390017070 con 0x7f9394103230 2024-04-25T11:41:49.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.730+0000 7f938b7fe640 1 --2- 172.21.15.83:0/714894811 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f93800a3c30 0x7f93800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:49.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.730+0000 7f939a17e640 1 --2- 172.21.15.83:0/714894811 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f93800a3c30 0x7f93800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:49.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.730+0000 7f938b7fe640 1 -- 172.21.15.83:0/714894811 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f93900cf880 con 0x7f9394103230 2024-04-25T11:41:49.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.731+0000 7f939a17e640 1 --2- 172.21.15.83:0/714894811 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f93800a3c30 0x7f93800a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9384009900 tx=0x7f93840314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:49.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.735+0000 7f938b7fe640 1 -- 172.21.15.83:0/714894811 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f939009bd60 con 0x7f9394103230 2024-04-25T11:41:49.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:49 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1104192340' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T11:41:49.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:49 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] up:active 2024-04-25T11:41:49.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:49 smithi083 ceph-mon[29330]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T11:41:49.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:49 smithi083 ceph-mon[29330]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T11:41:49.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:49 smithi083 ceph-mon[29330]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-25T11:41:49.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:49 smithi083 ceph-mon[29330]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T11:41:49.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:49 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/104935518' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T11:41:49.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.945+0000 7f939c3e2640 1 -- 172.21.15.83:0/714894811 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9394062390 con 0x7f93800a3c30 2024-04-25T11:41:49.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.970+0000 7f938b7fe640 1 -- 172.21.15.83:0/714894811 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9394062390 con 0x7f93800a3c30 2024-04-25T11:41:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.972+0000 7f939c3e2640 1 -- 172.21.15.83:0/714894811 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f93800a3c30 msgr2=0x7f93800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.972+0000 7f939c3e2640 1 --2- 172.21.15.83:0/714894811 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f93800a3c30 0x7f93800a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9384009900 tx=0x7f93840314a0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.972+0000 7f939c3e2640 1 -- 172.21.15.83:0/714894811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9394103230 msgr2=0x7f9394198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.972+0000 7f939c3e2640 1 --2- 172.21.15.83:0/714894811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9394103230 0x7f9394198810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f939000b4a0 tx=0x7f939000b960 comp rx=0 tx=0).stop 2024-04-25T11:41:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.972+0000 7f939c3e2640 1 -- 172.21.15.83:0/714894811 shutdown_connections 2024-04-25T11:41:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.972+0000 7f939c3e2640 1 --2- 172.21.15.83:0/714894811 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f93800a3c30 0x7f93800a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.972+0000 7f939c3e2640 1 --2- 172.21.15.83:0/714894811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9394103230 0x7f9394198810 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.972+0000 7f939c3e2640 1 --2- 172.21.15.83:0/714894811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9394103bf0 0x7f939419f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.972+0000 7f939c3e2640 1 --2- 172.21.15.83:0/714894811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9394100260 0x7f93941982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.972+0000 7f939c3e2640 1 -- 172.21.15.83:0/714894811 >> 172.21.15.83:0/714894811 conn(0x7f93940fbb90 msgr2=0x7f9394106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.972+0000 7f939c3e2640 1 -- 172.21.15.83:0/714894811 shutdown_connections 2024-04-25T11:41:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:49.972+0000 7f939c3e2640 1 -- 172.21.15.83:0/714894811 wait complete. 2024-04-25T11:41:49.971 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T11:41:50.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:49 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1104192340' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T11:41:50.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:49 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] up:active 2024-04-25T11:41:50.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:49 smithi094 ceph-mon[33032]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T11:41:50.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:49 smithi094 ceph-mon[33032]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T11:41:50.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:49 smithi094 ceph-mon[33032]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-25T11:41:50.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:49 smithi094 ceph-mon[33032]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T11:41:50.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:49 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/104935518' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T11:41:50.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:49 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1104192340' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T11:41:50.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:49 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] up:active 2024-04-25T11:41:50.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:49 smithi111 ceph-mon[32784]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T11:41:50.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:49 smithi111 ceph-mon[32784]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T11:41:50.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:49 smithi111 ceph-mon[32784]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-25T11:41:50.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:49 smithi111 ceph-mon[32784]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T11:41:50.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:49 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/104935518' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T11:41:50.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 -- 172.21.15.83:0/569451125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4254102a00 msgr2=0x7f4254109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 --2- 172.21.15.83:0/569451125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4254102a00 0x7f4254109320 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f424c009ca0 tx=0x7f424c02a8c0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 -- 172.21.15.83:0/569451125 shutdown_connections 2024-04-25T11:41:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 --2- 172.21.15.83:0/569451125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4254102a00 0x7f4254109320 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 --2- 172.21.15.83:0/569451125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4254102040 0x7f42541024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 --2- 172.21.15.83:0/569451125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4254100e60 0x7f4254101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 -- 172.21.15.83:0/569451125 >> 172.21.15.83:0/569451125 conn(0x7f42540fc440 msgr2=0x7f42540fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 -- 172.21.15.83:0/569451125 shutdown_connections 2024-04-25T11:41:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 -- 172.21.15.83:0/569451125 wait complete. 2024-04-25T11:41:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 Processor -- start 2024-04-25T11:41:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 -- start start 2024-04-25T11:41:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4254100e60 0x7f425406b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4254102040 0x7f425419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4254102a00 0x7f425419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f425410ca70 con 0x7f4254102040 2024-04-25T11:41:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f425410c8f0 con 0x7f4254100e60 2024-04-25T11:41:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.087+0000 7f425b795640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f425410cbf0 con 0x7f4254102a00 2024-04-25T11:41:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.088+0000 7f4258d30640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4254102040 0x7f425419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.088+0000 7f4258d30640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4254102040 0x7f425419cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43162/0 (socket says 172.21.15.83:43162) 2024-04-25T11:41:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.088+0000 7f4258d30640 1 -- 172.21.15.83:0/2676445838 learned_addr learned my addr 172.21.15.83:0/2676445838 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.088+0000 7f4259531640 1 --2- 172.21.15.83:0/2676445838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4254100e60 0x7f425406b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.088+0000 7f4258d30640 1 -- 172.21.15.83:0/2676445838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4254102a00 msgr2=0x7f425419f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.088+0000 7f4258d30640 1 --2- 172.21.15.83:0/2676445838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4254102a00 0x7f425419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.088+0000 7f4258d30640 1 -- 172.21.15.83:0/2676445838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4254100e60 msgr2=0x7f425406b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.088+0000 7f4258d30640 1 --2- 172.21.15.83:0/2676445838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4254100e60 0x7f425406b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.088+0000 7f4258d30640 1 -- 172.21.15.83:0/2676445838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425419fe30 con 0x7f4254102040 2024-04-25T11:41:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.088+0000 7f4258d30640 1 --2- 172.21.15.83:0/2676445838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4254102040 0x7f425419cf00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f424400d950 tx=0x7f424400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.088+0000 7f424a7fc640 1 -- 172.21.15.83:0/2676445838 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244017030 con 0x7f4254102040 2024-04-25T11:41:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.088+0000 7f425b795640 1 -- 172.21.15.83:0/2676445838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42541a0120 con 0x7f4254102040 2024-04-25T11:41:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.088+0000 7f424a7fc640 1 -- 172.21.15.83:0/2676445838 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4244004020 con 0x7f4254102040 2024-04-25T11:41:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.088+0000 7f424a7fc640 1 -- 172.21.15.83:0/2676445838 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244005560 con 0x7f4254102040 2024-04-25T11:41:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.088+0000 7f425b795640 1 -- 172.21.15.83:0/2676445838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42541a06c0 con 0x7f4254102040 2024-04-25T11:41:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.089+0000 7f425b795640 1 -- 172.21.15.83:0/2676445838 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4254108200 con 0x7f4254102040 2024-04-25T11:41:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.090+0000 7f424a7fc640 1 -- 172.21.15.83:0/2676445838 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4244009a30 con 0x7f4254102040 2024-04-25T11:41:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.091+0000 7f424a7fc640 1 --2- 172.21.15.83:0/2676445838 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42400a3bb0 0x7f42400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.091+0000 7f4259531640 1 --2- 172.21.15.83:0/2676445838 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42400a3bb0 0x7f42400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.091+0000 7f424a7fc640 1 -- 172.21.15.83:0/2676445838 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f42440cfcd0 con 0x7f4254102040 2024-04-25T11:41:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.091+0000 7f4259531640 1 --2- 172.21.15.83:0/2676445838 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42400a3bb0 0x7f42400a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4254101bb0 tx=0x7f4250009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:50.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.096+0000 7f424a7fc640 1 -- 172.21.15.83:0/2676445838 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f424400e350 con 0x7f4254102040 2024-04-25T11:41:50.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.300+0000 7f425b795640 1 -- 172.21.15.83:0/2676445838 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4254062390 con 0x7f42400a3bb0 2024-04-25T11:41:50.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.326+0000 7f424a7fc640 1 -- 172.21.15.83:0/2676445838 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4254062390 con 0x7f42400a3bb0 2024-04-25T11:41:50.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.328+0000 7f425b795640 1 -- 172.21.15.83:0/2676445838 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42400a3bb0 msgr2=0x7f42400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.328+0000 7f425b795640 1 --2- 172.21.15.83:0/2676445838 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42400a3bb0 0x7f42400a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4254101bb0 tx=0x7f4250009110 comp rx=0 tx=0).stop 2024-04-25T11:41:50.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.328+0000 7f425b795640 1 -- 172.21.15.83:0/2676445838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4254102040 msgr2=0x7f425419cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.328+0000 7f425b795640 1 --2- 172.21.15.83:0/2676445838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4254102040 0x7f425419cf00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f424400d950 tx=0x7f424400dd50 comp rx=0 tx=0).stop 2024-04-25T11:41:50.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.328+0000 7f425b795640 1 -- 172.21.15.83:0/2676445838 shutdown_connections 2024-04-25T11:41:50.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.328+0000 7f425b795640 1 --2- 172.21.15.83:0/2676445838 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42400a3bb0 0x7f42400a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.328+0000 7f425b795640 1 --2- 172.21.15.83:0/2676445838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4254102040 0x7f425419cf00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.328+0000 7f425b795640 1 --2- 172.21.15.83:0/2676445838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4254102a00 0x7f425419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.328+0000 7f425b795640 1 --2- 172.21.15.83:0/2676445838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4254100e60 0x7f425406b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.328+0000 7f425b795640 1 -- 172.21.15.83:0/2676445838 >> 172.21.15.83:0/2676445838 conn(0x7f42540fc440 msgr2=0x7f42541053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.328+0000 7f425b795640 1 -- 172.21.15.83:0/2676445838 shutdown_connections 2024-04-25T11:41:50.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.328+0000 7f425b795640 1 -- 172.21.15.83:0/2676445838 wait complete. 2024-04-25T11:41:50.325 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:41:49 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:41:49.950+0000 7f9622556700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:41:50.325 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:41:49 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:41:49.950+0000 7f9622556700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:41:50.325 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:41:49 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:41:49.950+0000 7f9622556700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:41:50.325 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:41:49 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:41:49.950+0000 7f9622556700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:41:50.325 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:41:49 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:41:49.950+0000 7f9622556700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:41:50.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T11:41:50.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.443+0000 7f284856a640 1 -- 172.21.15.83:0/2318852402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2840101fa0 msgr2=0x7f28401023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.443+0000 7f284856a640 1 --2- 172.21.15.83:0/2318852402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2840101fa0 0x7f28401023c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f283c009900 tx=0x7f283c02a410 comp rx=0 tx=0).stop 2024-04-25T11:41:50.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.443+0000 7f284856a640 1 -- 172.21.15.83:0/2318852402 shutdown_connections 2024-04-25T11:41:50.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.443+0000 7f284856a640 1 --2- 172.21.15.83:0/2318852402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2840103b50 0x7f284010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.443+0000 7f284856a640 1 --2- 172.21.15.83:0/2318852402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2840103190 0x7f2840103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.443+0000 7f284856a640 1 --2- 172.21.15.83:0/2318852402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2840101fa0 0x7f28401023c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.443+0000 7f284856a640 1 -- 172.21.15.83:0/2318852402 >> 172.21.15.83:0/2318852402 conn(0x7f28400fd530 msgr2=0x7f28400ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:50.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.443+0000 7f284856a640 1 -- 172.21.15.83:0/2318852402 shutdown_connections 2024-04-25T11:41:50.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.443+0000 7f284856a640 1 -- 172.21.15.83:0/2318852402 wait complete. 2024-04-25T11:41:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.443+0000 7f284856a640 1 Processor -- start 2024-04-25T11:41:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.443+0000 7f284856a640 1 -- start start 2024-04-25T11:41:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f284856a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2840101fa0 0x7f2840075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f284856a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2840103190 0x7f2840074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f284856a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2840103b50 0x7f2840074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f284856a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f284010dae0 con 0x7f2840103b50 2024-04-25T11:41:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f284856a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f284010d960 con 0x7f2840103190 2024-04-25T11:41:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f284856a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f284010dc60 con 0x7f2840101fa0 2024-04-25T11:41:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f2846b07640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2840103b50 0x7f2840074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f2846b07640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2840103b50 0x7f2840074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43190/0 (socket says 172.21.15.83:43190) 2024-04-25T11:41:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f2846b07640 1 -- 172.21.15.83:0/3774121344 learned_addr learned my addr 172.21.15.83:0/3774121344 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f2846306640 1 --2- 172.21.15.83:0/3774121344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2840101fa0 0x7f2840075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f2845b05640 1 --2- 172.21.15.83:0/3774121344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2840103190 0x7f2840074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f2846b07640 1 -- 172.21.15.83:0/3774121344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2840101fa0 msgr2=0x7f2840075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f2846b07640 1 --2- 172.21.15.83:0/3774121344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2840101fa0 0x7f2840075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f2846b07640 1 -- 172.21.15.83:0/3774121344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2840103190 msgr2=0x7f2840074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f2846b07640 1 --2- 172.21.15.83:0/3774121344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2840103190 0x7f2840074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f2846b07640 1 -- 172.21.15.83:0/3774121344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2840075110 con 0x7f2840103b50 2024-04-25T11:41:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f2846b07640 1 --2- 172.21.15.83:0/3774121344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2840103b50 0x7f2840074880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f283800eb10 tx=0x7f283800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f28377fe640 1 -- 172.21.15.83:0/3774121344 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838004b20 con 0x7f2840103b50 2024-04-25T11:41:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f28377fe640 1 -- 172.21.15.83:0/3774121344 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2838004cc0 con 0x7f2840103b50 2024-04-25T11:41:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f284856a640 1 -- 172.21.15.83:0/3774121344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28401a2350 con 0x7f2840103b50 2024-04-25T11:41:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f28377fe640 1 -- 172.21.15.83:0/3774121344 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2838010670 con 0x7f2840103b50 2024-04-25T11:41:50.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.444+0000 7f284856a640 1 -- 172.21.15.83:0/3774121344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28401a2850 con 0x7f2840103b50 2024-04-25T11:41:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.445+0000 7f284856a640 1 -- 172.21.15.83:0/3774121344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2840109400 con 0x7f2840103b50 2024-04-25T11:41:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.446+0000 7f28377fe640 1 -- 172.21.15.83:0/3774121344 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2838005020 con 0x7f2840103b50 2024-04-25T11:41:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.447+0000 7f28377fe640 1 --2- 172.21.15.83:0/3774121344 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f282c0a3bb0 0x7f282c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.447+0000 7f2846306640 1 --2- 172.21.15.83:0/3774121344 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f282c0a3bb0 0x7f282c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.447+0000 7f28377fe640 1 -- 172.21.15.83:0/3774121344 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f28380d08a0 con 0x7f2840103b50 2024-04-25T11:41:50.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.447+0000 7f2846306640 1 --2- 172.21.15.83:0/3774121344 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f282c0a3bb0 0x7f282c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f283c009230 tx=0x7f283c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:50.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.451+0000 7f28377fe640 1 -- 172.21.15.83:0/3774121344 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f283809d3d0 con 0x7f2840103b50 2024-04-25T11:41:50.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.664+0000 7f284856a640 1 -- 172.21.15.83:0/3774121344 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f2840062420 con 0x7f282c0a3bb0 2024-04-25T11:41:50.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.670+0000 7f28377fe640 1 -- 172.21.15.83:0/3774121344 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2840062420 con 0x7f282c0a3bb0 2024-04-25T11:41:50.656 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/qa/sv_0/d9155369-188c-430c-a378-38556e1a74ee 2024-04-25T11:41:50.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.671+0000 7f284856a640 1 -- 172.21.15.83:0/3774121344 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f282c0a3bb0 msgr2=0x7f282c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.671+0000 7f284856a640 1 --2- 172.21.15.83:0/3774121344 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f282c0a3bb0 0x7f282c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f283c009230 tx=0x7f283c0314a0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.671+0000 7f284856a640 1 -- 172.21.15.83:0/3774121344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2840103b50 msgr2=0x7f2840074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.671+0000 7f284856a640 1 --2- 172.21.15.83:0/3774121344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2840103b50 0x7f2840074880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f283800eb10 tx=0x7f283800c2d0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.671+0000 7f284856a640 1 -- 172.21.15.83:0/3774121344 shutdown_connections 2024-04-25T11:41:50.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.671+0000 7f284856a640 1 --2- 172.21.15.83:0/3774121344 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f282c0a3bb0 0x7f282c0a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.671+0000 7f284856a640 1 --2- 172.21.15.83:0/3774121344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2840103b50 0x7f2840074880 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.671+0000 7f284856a640 1 --2- 172.21.15.83:0/3774121344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2840103190 0x7f2840074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.671+0000 7f284856a640 1 --2- 172.21.15.83:0/3774121344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2840101fa0 0x7f2840075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.672+0000 7f284856a640 1 -- 172.21.15.83:0/3774121344 >> 172.21.15.83:0/3774121344 conn(0x7f28400fd530 msgr2=0x7f28401065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:50.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.672+0000 7f284856a640 1 -- 172.21.15.83:0/3774121344 shutdown_connections 2024-04-25T11:41:50.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.672+0000 7f284856a640 1 -- 172.21.15.83:0/3774121344 wait complete. 2024-04-25T11:41:50.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T11:41:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.785+0000 7f408bb76640 1 -- 172.21.15.83:0/2063484973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4084105340 msgr2=0x7f4084107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.785+0000 7f408bb76640 1 --2- 172.21.15.83:0/2063484973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4084105340 0x7f4084107730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4074009900 tx=0x7f407402a410 comp rx=0 tx=0).stop 2024-04-25T11:41:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.785+0000 7f408bb76640 1 -- 172.21.15.83:0/2063484973 shutdown_connections 2024-04-25T11:41:50.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.785+0000 7f408bb76640 1 --2- 172.21.15.83:0/2063484973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4084105340 0x7f4084107730 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.785+0000 7f408bb76640 1 --2- 172.21.15.83:0/2063484973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4084107c70 0x7f408410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.785+0000 7f408bb76640 1 --2- 172.21.15.83:0/2063484973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4084102a10 0x7f4084104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.785+0000 7f408bb76640 1 -- 172.21.15.83:0/2063484973 >> 172.21.15.83:0/2063484973 conn(0x7f40840fc400 msgr2=0x7f40840fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.786+0000 7f408bb76640 1 -- 172.21.15.83:0/2063484973 shutdown_connections 2024-04-25T11:41:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.786+0000 7f408bb76640 1 -- 172.21.15.83:0/2063484973 wait complete. 2024-04-25T11:41:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.786+0000 7f408bb76640 1 Processor -- start 2024-04-25T11:41:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.786+0000 7f408bb76640 1 -- start start 2024-04-25T11:41:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.786+0000 7f408bb76640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4084102a10 0x7f4084198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.786+0000 7f408bb76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4084105340 0x7f4084198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.786+0000 7f408bb76640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4084107c70 0x7f408419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.786+0000 7f408bb76640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f408410dac0 con 0x7f4084105340 2024-04-25T11:41:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.786+0000 7f408bb76640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f408410d940 con 0x7f4084102a10 2024-04-25T11:41:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.786+0000 7f408bb76640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f408410dc40 con 0x7f4084107c70 2024-04-25T11:41:50.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.786+0000 7f4089111640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4084105340 0x7f4084198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.786+0000 7f4089111640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4084105340 0x7f4084198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43208/0 (socket says 172.21.15.83:43208) 2024-04-25T11:41:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.786+0000 7f4089111640 1 -- 172.21.15.83:0/2250548421 learned_addr learned my addr 172.21.15.83:0/2250548421 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.786+0000 7f4089912640 1 --2- 172.21.15.83:0/2250548421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4084102a10 0x7f4084198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.786+0000 7f4089111640 1 -- 172.21.15.83:0/2250548421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4084107c70 msgr2=0x7f408419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.787+0000 7f4089111640 1 --2- 172.21.15.83:0/2250548421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4084107c70 0x7f408419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.787+0000 7f4089111640 1 -- 172.21.15.83:0/2250548421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4084102a10 msgr2=0x7f4084198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.787+0000 7f4089111640 1 --2- 172.21.15.83:0/2250548421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4084102a10 0x7f4084198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.787+0000 7f4089111640 1 -- 172.21.15.83:0/2250548421 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408419fe20 con 0x7f4084105340 2024-04-25T11:41:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.787+0000 7f4089111640 1 --2- 172.21.15.83:0/2250548421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4084105340 0x7f4084198740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4074006010 tx=0x7f40740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.787+0000 7f407affd640 1 -- 172.21.15.83:0/2250548421 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407403f030 con 0x7f4084105340 2024-04-25T11:41:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.787+0000 7f407affd640 1 -- 172.21.15.83:0/2250548421 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40740318f0 con 0x7f4084105340 2024-04-25T11:41:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.787+0000 7f408bb76640 1 -- 172.21.15.83:0/2250548421 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40841a00b0 con 0x7f4084105340 2024-04-25T11:41:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.787+0000 7f407affd640 1 -- 172.21.15.83:0/2250548421 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4074035070 con 0x7f4084105340 2024-04-25T11:41:50.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.787+0000 7f408bb76640 1 -- 172.21.15.83:0/2250548421 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40841a05f0 con 0x7f4084105340 2024-04-25T11:41:50.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.788+0000 7f408bb76640 1 -- 172.21.15.83:0/2250548421 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40840fdef0 con 0x7f4084105340 2024-04-25T11:41:50.777 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:41:50 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b[63285]: 2024-04-25T11:41:50.423+0000 7fa072f29700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:41:50.777 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:41:50 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b[63285]: 2024-04-25T11:41:50.423+0000 7fa072f29700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T11:41:50.777 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:41:50 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b[63285]: 2024-04-25T11:41:50.423+0000 7fa072f29700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T11:41:50.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:50 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/104935518' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T11:41:50.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:50 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] up:active 2024-04-25T11:41:50.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:50 smithi094 ceph-mon[33032]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T11:41:50.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:50 smithi094 ceph-mon[33032]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-25T11:41:50.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:50 smithi094 ceph-mon[33032]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-04-25T11:41:50.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:50 smithi094 ceph-mon[33032]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-04-25T11:41:50.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:50 smithi094 ceph-mon[33032]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 12 op/s 2024-04-25T11:41:50.778 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:50 smithi094 ceph-mon[33032]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:50.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.789+0000 7f407affd640 1 -- 172.21.15.83:0/2250548421 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4074042030 con 0x7f4084105340 2024-04-25T11:41:50.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.789+0000 7f407affd640 1 --2- 172.21.15.83:0/2250548421 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f40700a3c30 0x7f40700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:50.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.790+0000 7f4089912640 1 --2- 172.21.15.83:0/2250548421 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f40700a3c30 0x7f40700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:50.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.790+0000 7f407affd640 1 -- 172.21.15.83:0/2250548421 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f40740f07d0 con 0x7f4084105340 2024-04-25T11:41:50.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.790+0000 7f4089912640 1 --2- 172.21.15.83:0/2250548421 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f40700a3c30 0x7f40700a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4080005fd0 tx=0x7f408000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:50.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:50.794+0000 7f407affd640 1 -- 172.21.15.83:0/2250548421 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40740bd300 con 0x7f4084105340 2024-04-25T11:41:50.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:50 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/104935518' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T11:41:50.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:50 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] up:active 2024-04-25T11:41:50.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:50 smithi083 ceph-mon[29330]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T11:41:50.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:50 smithi083 ceph-mon[29330]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-25T11:41:50.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:50 smithi083 ceph-mon[29330]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-04-25T11:41:50.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:50 smithi083 ceph-mon[29330]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-04-25T11:41:50.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:50 smithi083 ceph-mon[29330]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 12 op/s 2024-04-25T11:41:50.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:50 smithi083 ceph-mon[29330]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:50.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.000+0000 7f408bb76640 1 -- 172.21.15.83:0/2250548421 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4084062420 con 0x7f40700a3c30 2024-04-25T11:41:51.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.025+0000 7f407affd640 1 -- 172.21.15.83:0/2250548421 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4084062420 con 0x7f40700a3c30 2024-04-25T11:41:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.027+0000 7f408bb76640 1 -- 172.21.15.83:0/2250548421 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f40700a3c30 msgr2=0x7f40700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.027+0000 7f408bb76640 1 --2- 172.21.15.83:0/2250548421 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f40700a3c30 0x7f40700a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4080005fd0 tx=0x7f408000b390 comp rx=0 tx=0).stop 2024-04-25T11:41:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.027+0000 7f408bb76640 1 -- 172.21.15.83:0/2250548421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4084105340 msgr2=0x7f4084198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.027+0000 7f408bb76640 1 --2- 172.21.15.83:0/2250548421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4084105340 0x7f4084198740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4074006010 tx=0x7f40740042f0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.028+0000 7f408bb76640 1 -- 172.21.15.83:0/2250548421 shutdown_connections 2024-04-25T11:41:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.028+0000 7f408bb76640 1 --2- 172.21.15.83:0/2250548421 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f40700a3c30 0x7f40700a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.028+0000 7f408bb76640 1 --2- 172.21.15.83:0/2250548421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4084105340 0x7f4084198740 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.028+0000 7f408bb76640 1 --2- 172.21.15.83:0/2250548421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4084107c70 0x7f408419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.028+0000 7f408bb76640 1 --2- 172.21.15.83:0/2250548421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4084102a10 0x7f4084198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.028+0000 7f408bb76640 1 -- 172.21.15.83:0/2250548421 >> 172.21.15.83:0/2250548421 conn(0x7f40840fc400 msgr2=0x7f40841048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.028+0000 7f408bb76640 1 -- 172.21.15.83:0/2250548421 shutdown_connections 2024-04-25T11:41:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.028+0000 7f408bb76640 1 -- 172.21.15.83:0/2250548421 wait complete. 2024-04-25T11:41:51.025 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T11:41:51.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:50 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/104935518' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T11:41:51.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:50 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] up:active 2024-04-25T11:41:51.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:50 smithi111 ceph-mon[32784]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-25T11:41:51.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:50 smithi111 ceph-mon[32784]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-25T11:41:51.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:50 smithi111 ceph-mon[32784]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-04-25T11:41:51.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:50 smithi111 ceph-mon[32784]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-04-25T11:41:51.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:50 smithi111 ceph-mon[32784]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 12 op/s 2024-04-25T11:41:51.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:50 smithi111 ceph-mon[32784]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.142+0000 7f677f674640 1 -- 172.21.15.83:0/1782415970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6778102a30 msgr2=0x7f6778104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.142+0000 7f677f674640 1 --2- 172.21.15.83:0/1782415970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6778102a30 0x7f6778104e20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6774009900 tx=0x7f677402a410 comp rx=0 tx=0).stop 2024-04-25T11:41:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.142+0000 7f677f674640 1 -- 172.21.15.83:0/1782415970 shutdown_connections 2024-04-25T11:41:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.142+0000 7f677f674640 1 --2- 172.21.15.83:0/1782415970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6778107c90 0x7f677810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.142+0000 7f677f674640 1 --2- 172.21.15.83:0/1782415970 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6778105360 0x7f6778107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.142+0000 7f677f674640 1 --2- 172.21.15.83:0/1782415970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6778102a30 0x7f6778104e20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.142+0000 7f677f674640 1 -- 172.21.15.83:0/1782415970 >> 172.21.15.83:0/1782415970 conn(0x7f67780fc420 msgr2=0x7f67780fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.142+0000 7f677f674640 1 -- 172.21.15.83:0/1782415970 shutdown_connections 2024-04-25T11:41:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.142+0000 7f677f674640 1 -- 172.21.15.83:0/1782415970 wait complete. 2024-04-25T11:41:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677f674640 1 Processor -- start 2024-04-25T11:41:51.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677f674640 1 -- start start 2024-04-25T11:41:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677f674640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6778102a30 0x7f6778198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677f674640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6778105360 0x7f6778198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677f674640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6778107c90 0x7f677819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677f674640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f677810daf0 con 0x7f6778102a30 2024-04-25T11:41:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677f674640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f677810d970 con 0x7f6778107c90 2024-04-25T11:41:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677f674640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f677810dc70 con 0x7f6778105360 2024-04-25T11:41:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677d410640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6778102a30 0x7f6778198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677d410640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6778102a30 0x7f6778198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43236/0 (socket says 172.21.15.83:43236) 2024-04-25T11:41:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677d410640 1 -- 172.21.15.83:0/3214728873 learned_addr learned my addr 172.21.15.83:0/3214728873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677dc11640 1 --2- 172.21.15.83:0/3214728873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6778107c90 0x7f677819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677cc0f640 1 --2- 172.21.15.83:0/3214728873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6778105360 0x7f6778198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677d410640 1 -- 172.21.15.83:0/3214728873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6778105360 msgr2=0x7f6778198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677d410640 1 --2- 172.21.15.83:0/3214728873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6778105360 0x7f6778198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677d410640 1 -- 172.21.15.83:0/3214728873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6778107c90 msgr2=0x7f677819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677d410640 1 --2- 172.21.15.83:0/3214728873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6778107c90 0x7f677819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677d410640 1 -- 172.21.15.83:0/3214728873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677819fe40 con 0x7f6778102a30 2024-04-25T11:41:51.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677cc0f640 1 --2- 172.21.15.83:0/3214728873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6778105360 0x7f6778198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:51.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f677d410640 1 --2- 172.21.15.83:0/3214728873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6778102a30 0x7f6778198220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6774005940 tx=0x7f6774004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:51.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.143+0000 7f676e7fc640 1 -- 172.21.15.83:0/3214728873 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677403f030 con 0x7f6778102a30 2024-04-25T11:41:51.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.144+0000 7f676e7fc640 1 -- 172.21.15.83:0/3214728873 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6774003cb0 con 0x7f6778102a30 2024-04-25T11:41:51.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.144+0000 7f677f674640 1 -- 172.21.15.83:0/3214728873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67781a00d0 con 0x7f6778102a30 2024-04-25T11:41:51.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.144+0000 7f676e7fc640 1 -- 172.21.15.83:0/3214728873 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6774035070 con 0x7f6778102a30 2024-04-25T11:41:51.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.144+0000 7f677f674640 1 -- 172.21.15.83:0/3214728873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67781a0610 con 0x7f6778102a30 2024-04-25T11:41:51.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.144+0000 7f677f674640 1 -- 172.21.15.83:0/3214728873 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67780fdf10 con 0x7f6778102a30 2024-04-25T11:41:51.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.146+0000 7f676e7fc640 1 -- 172.21.15.83:0/3214728873 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6774042030 con 0x7f6778102a30 2024-04-25T11:41:51.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.146+0000 7f676e7fc640 1 --2- 172.21.15.83:0/3214728873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f67640a3c30 0x7f67640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.146+0000 7f677cc0f640 1 --2- 172.21.15.83:0/3214728873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f67640a3c30 0x7f67640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.146+0000 7f676e7fc640 1 -- 172.21.15.83:0/3214728873 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f67740f0b80 con 0x7f6778102a30 2024-04-25T11:41:51.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.146+0000 7f677cc0f640 1 --2- 172.21.15.83:0/3214728873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f67640a3c30 0x7f67640a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6768005d00 tx=0x7f6768005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:51.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.151+0000 7f676e7fc640 1 -- 172.21.15.83:0/3214728873 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6774003e60 con 0x7f6778102a30 2024-04-25T11:41:51.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.355+0000 7f677f674640 1 -- 172.21.15.83:0/3214728873 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f6778062420 con 0x7f67640a3c30 2024-04-25T11:41:51.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.361+0000 7f676e7fc640 1 -- 172.21.15.83:0/3214728873 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f6778062420 con 0x7f67640a3c30 2024-04-25T11:41:51.347 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/qa/sv_1/470219e6-04e0-4a93-9a4c-4bdb75bfa50d 2024-04-25T11:41:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.363+0000 7f677f674640 1 -- 172.21.15.83:0/3214728873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f67640a3c30 msgr2=0x7f67640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.363+0000 7f677f674640 1 --2- 172.21.15.83:0/3214728873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f67640a3c30 0x7f67640a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6768005d00 tx=0x7f6768005c70 comp rx=0 tx=0).stop 2024-04-25T11:41:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.363+0000 7f677f674640 1 -- 172.21.15.83:0/3214728873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6778102a30 msgr2=0x7f6778198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.363+0000 7f677f674640 1 --2- 172.21.15.83:0/3214728873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6778102a30 0x7f6778198220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6774005940 tx=0x7f6774004430 comp rx=0 tx=0).stop 2024-04-25T11:41:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.363+0000 7f677f674640 1 -- 172.21.15.83:0/3214728873 shutdown_connections 2024-04-25T11:41:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.363+0000 7f677f674640 1 --2- 172.21.15.83:0/3214728873 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f67640a3c30 0x7f67640a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.363+0000 7f677f674640 1 --2- 172.21.15.83:0/3214728873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6778107c90 0x7f677819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.363+0000 7f677f674640 1 --2- 172.21.15.83:0/3214728873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6778105360 0x7f6778198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.363+0000 7f677f674640 1 --2- 172.21.15.83:0/3214728873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6778102a30 0x7f6778198220 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.363+0000 7f677f674640 1 -- 172.21.15.83:0/3214728873 >> 172.21.15.83:0/3214728873 conn(0x7f67780fc420 msgr2=0x7f6778103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.363+0000 7f677f674640 1 -- 172.21.15.83:0/3214728873 shutdown_connections 2024-04-25T11:41:51.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.363+0000 7f677f674640 1 -- 172.21.15.83:0/3214728873 wait complete. 2024-04-25T11:41:51.361 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:41:51.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.475+0000 7fddc489a640 1 -- 172.21.15.83:0/3903726859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc0101fe0 msgr2=0x7fddc0102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.475+0000 7fddc489a640 1 --2- 172.21.15.83:0/3903726859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc0101fe0 0x7fddc0102460 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009900 tx=0x7fddb002a410 comp rx=0 tx=0).stop 2024-04-25T11:41:51.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.476+0000 7fddc489a640 1 -- 172.21.15.83:0/3903726859 shutdown_connections 2024-04-25T11:41:51.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.476+0000 7fddc489a640 1 --2- 172.21.15.83:0/3903726859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc0101fe0 0x7fddc0102460 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.476+0000 7fddc489a640 1 --2- 172.21.15.83:0/3903726859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddc01029a0 0x7fddc01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.476+0000 7fddc489a640 1 --2- 172.21.15.83:0/3903726859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddc0100e90 0x7fddc01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.476+0000 7fddc489a640 1 -- 172.21.15.83:0/3903726859 >> 172.21.15.83:0/3903726859 conn(0x7fddc00fc400 msgr2=0x7fddc00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.476+0000 7fddc489a640 1 -- 172.21.15.83:0/3903726859 shutdown_connections 2024-04-25T11:41:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.476+0000 7fddc489a640 1 -- 172.21.15.83:0/3903726859 wait complete. 2024-04-25T11:41:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.476+0000 7fddc489a640 1 Processor -- start 2024-04-25T11:41:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.476+0000 7fddc489a640 1 -- start start 2024-04-25T11:41:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.476+0000 7fddc489a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddc0100e90 0x7fddc0104fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.476+0000 7fddc489a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc0101fe0 0x7fddc01035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.476+0000 7fddc489a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddc01029a0 0x7fddc0103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.476+0000 7fddc489a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fddc010c9f0 con 0x7fddc0101fe0 2024-04-25T11:41:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddc489a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fddc010c870 con 0x7fddc0100e90 2024-04-25T11:41:51.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddc489a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fddc010cb70 con 0x7fddc01029a0 2024-04-25T11:41:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddbdd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc0101fe0 0x7fddc01035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddbe59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddc0100e90 0x7fddc0104fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddbdd9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc0101fe0 0x7fddc01035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43260/0 (socket says 172.21.15.83:43260) 2024-04-25T11:41:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddbdd9b640 1 -- 172.21.15.83:0/3842458307 learned_addr learned my addr 172.21.15.83:0/3842458307 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddbed9d640 1 --2- 172.21.15.83:0/3842458307 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddc01029a0 0x7fddc0103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddbe59c640 1 -- 172.21.15.83:0/3842458307 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddc01029a0 msgr2=0x7fddc0103b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddbe59c640 1 --2- 172.21.15.83:0/3842458307 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddc01029a0 0x7fddc0103b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddbe59c640 1 -- 172.21.15.83:0/3842458307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc0101fe0 msgr2=0x7fddc01035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddbe59c640 1 --2- 172.21.15.83:0/3842458307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc0101fe0 0x7fddc01035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddbe59c640 1 -- 172.21.15.83:0/3842458307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc01043c0 con 0x7fddc0100e90 2024-04-25T11:41:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddbdd9b640 1 --2- 172.21.15.83:0/3842458307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc0101fe0 0x7fddc01035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:41:51.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddbe59c640 1 --2- 172.21.15.83:0/3842458307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddc0100e90 0x7fddc0104fb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fddb400c970 tx=0x7fddb400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddaf7fe640 1 -- 172.21.15.83:0/3842458307 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb4007bb0 con 0x7fddc0100e90 2024-04-25T11:41:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddaf7fe640 1 -- 172.21.15.83:0/3842458307 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddb4007d50 con 0x7fddc0100e90 2024-04-25T11:41:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddc489a640 1 -- 172.21.15.83:0/3842458307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc01a22f0 con 0x7fddc0100e90 2024-04-25T11:41:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddaf7fe640 1 -- 172.21.15.83:0/3842458307 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb4004df0 con 0x7fddc0100e90 2024-04-25T11:41:51.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.477+0000 7fddc489a640 1 -- 172.21.15.83:0/3842458307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc01a2840 con 0x7fddc0100e90 2024-04-25T11:41:51.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.478+0000 7fddc489a640 1 -- 172.21.15.83:0/3842458307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc01a2b60 con 0x7fddc0100e90 2024-04-25T11:41:51.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.479+0000 7fddaf7fe640 1 -- 172.21.15.83:0/3842458307 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fddb4005020 con 0x7fddc0100e90 2024-04-25T11:41:51.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.480+0000 7fddaf7fe640 1 --2- 172.21.15.83:0/3842458307 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdda80a3c30 0x7fdda80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.480+0000 7fddbdd9b640 1 --2- 172.21.15.83:0/3842458307 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdda80a3c30 0x7fdda80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.480+0000 7fddaf7fe640 1 -- 172.21.15.83:0/3842458307 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fddb40d0370 con 0x7fddc0100e90 2024-04-25T11:41:51.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.480+0000 7fddbdd9b640 1 --2- 172.21.15.83:0/3842458307 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdda80a3c30 0x7fdda80a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fddb0032fd0 tx=0x7fddb00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:51.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.484+0000 7fddaf7fe640 1 -- 172.21.15.83:0/3842458307 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fddb4014030 con 0x7fddc0100e90 2024-04-25T11:41:51.642 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:51 smithi083 ceph-mon[29330]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:51.642 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:51 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] up:active 2024-04-25T11:41:51.642 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:51 smithi083 ceph-mon[29330]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T11:41:51.642 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:51 smithi083 ceph-mon[29330]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T11:41:51.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:51 smithi083 ceph-mon[29330]: Cluster is now healthy 2024-04-25T11:41:51.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:51 smithi083 ceph-mon[29330]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-04-25T11:41:51.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:51 smithi083 ceph-mon[29330]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-25T11:41:51.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:51 smithi083 ceph-mon[29330]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-04-25T11:41:51.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:51 smithi083 ceph-mon[29330]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:51.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:51 smithi083 ceph-mon[29330]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:51.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:51 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] up:active 2024-04-25T11:41:51.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:51 smithi083 ceph-mon[29330]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T11:41:51.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.711+0000 7fddc489a640 1 -- 172.21.15.83:0/3842458307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddc01a2d30 con 0x7fddc0100e90 2024-04-25T11:41:51.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.712+0000 7fddaf7fe640 1 -- 172.21.15.83:0/3842458307 <== mon.2 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7fddb409cf00 con 0x7fddc0100e90 2024-04-25T11:41:51.698 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:51.699 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:41:44.787926+0000","modified":"2024-04-25T11:41:51.578220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24469,"mds_2":24442,"mds_3":24440,"mds_4":24430},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/1568064835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1568064835},{"type":"v1","addr":"172.21.15.83:6839","nonce":1568064835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/1576544313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1576544313},{"type":"v1","addr":"172.21.15.83:6841","nonce":1576544313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":4,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.94:6837/2475976592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2475976592},{"type":"v1","addr":"172.21.15.94:6837","nonce":2475976592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.111:6837/2927430878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2927430878},{"type":"v1","addr":"172.21.15.111:6837","nonce":2927430878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/2233531151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2233531151},{"type":"v1","addr":"172.21.15.111:6839","nonce":2233531151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/1461078471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1461078471},{"type":"v1","addr":"172.21.15.94:6839","nonce":1461078471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24440,24430,24479,24469,24442]},"id":1}]} 2024-04-25T11:41:51.699 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-04-25T11:41:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.714+0000 7fddc489a640 1 -- 172.21.15.83:0/3842458307 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdda80a3c30 msgr2=0x7fdda80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.714+0000 7fddc489a640 1 --2- 172.21.15.83:0/3842458307 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdda80a3c30 0x7fdda80a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fddb0032fd0 tx=0x7fddb00314a0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.714+0000 7fddc489a640 1 -- 172.21.15.83:0/3842458307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddc0100e90 msgr2=0x7fddc0104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.714+0000 7fddc489a640 1 --2- 172.21.15.83:0/3842458307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddc0100e90 0x7fddc0104fb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fddb400c970 tx=0x7fddb400ce30 comp rx=0 tx=0).stop 2024-04-25T11:41:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.714+0000 7fddc489a640 1 -- 172.21.15.83:0/3842458307 shutdown_connections 2024-04-25T11:41:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.714+0000 7fddc489a640 1 --2- 172.21.15.83:0/3842458307 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdda80a3c30 0x7fdda80a60f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.714+0000 7fddc489a640 1 --2- 172.21.15.83:0/3842458307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fddc0101fe0 0x7fddc01035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.714+0000 7fddc489a640 1 --2- 172.21.15.83:0/3842458307 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fddc01029a0 0x7fddc0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.714+0000 7fddc489a640 1 --2- 172.21.15.83:0/3842458307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fddc0100e90 0x7fddc0104fb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.714+0000 7fddc489a640 1 -- 172.21.15.83:0/3842458307 >> 172.21.15.83:0/3842458307 conn(0x7fddc00fc400 msgr2=0x7fddc00690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.714+0000 7fddc489a640 1 -- 172.21.15.83:0/3842458307 shutdown_connections 2024-04-25T11:41:51.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.714+0000 7fddc489a640 1 -- 172.21.15.83:0/3842458307 wait complete. 2024-04-25T11:41:51.712 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:41:51.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.826+0000 7f84b71ad640 1 -- 172.21.15.83:0/2975295536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b0103180 msgr2=0x7f84b01055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.826+0000 7f84b71ad640 1 --2- 172.21.15.83:0/2975295536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b0103180 0x7f84b01055f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f849c009900 tx=0x7f849c02a410 comp rx=0 tx=0).stop 2024-04-25T11:41:51.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.826+0000 7f84b71ad640 1 -- 172.21.15.83:0/2975295536 shutdown_connections 2024-04-25T11:41:51.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.826+0000 7f84b71ad640 1 --2- 172.21.15.83:0/2975295536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b0103180 0x7f84b01055f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.826+0000 7f84b71ad640 1 --2- 172.21.15.83:0/2975295536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84b0105b30 0x7f84b0105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.826+0000 7f84b71ad640 1 --2- 172.21.15.83:0/2975295536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b0100810 0x7f84b0102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.826+0000 7f84b71ad640 1 -- 172.21.15.83:0/2975295536 >> 172.21.15.83:0/2975295536 conn(0x7f84b00fc450 msgr2=0x7f84b00fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.826+0000 7f84b71ad640 1 -- 172.21.15.83:0/2975295536 shutdown_connections 2024-04-25T11:41:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.826+0000 7f84b71ad640 1 -- 172.21.15.83:0/2975295536 wait complete. 2024-04-25T11:41:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b71ad640 1 Processor -- start 2024-04-25T11:41:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b71ad640 1 -- start start 2024-04-25T11:41:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b71ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b0100810 0x7f84b019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b71ad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84b0103180 0x7f84b019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b71ad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b0105b30 0x7f84b01a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b71ad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84b01140c0 con 0x7f84b0100810 2024-04-25T11:41:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b71ad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f84b0113f40 con 0x7f84b0105b30 2024-04-25T11:41:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b71ad640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f84b0114240 con 0x7f84b0103180 2024-04-25T11:41:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b4f49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b0100810 0x7f84b019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b4f49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b0100810 0x7f84b019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43280/0 (socket says 172.21.15.83:43280) 2024-04-25T11:41:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b4f49640 1 -- 172.21.15.83:0/3373894701 learned_addr learned my addr 172.21.15.83:0/3373894701 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:51.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b574a640 1 --2- 172.21.15.83:0/3373894701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b0105b30 0x7f84b01a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84affff640 1 --2- 172.21.15.83:0/3373894701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84b0103180 0x7f84b019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b4f49640 1 -- 172.21.15.83:0/3373894701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84b0103180 msgr2=0x7f84b019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b4f49640 1 --2- 172.21.15.83:0/3373894701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84b0103180 0x7f84b019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b4f49640 1 -- 172.21.15.83:0/3373894701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b0105b30 msgr2=0x7f84b01a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b4f49640 1 --2- 172.21.15.83:0/3373894701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b0105b30 0x7f84b01a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b4f49640 1 -- 172.21.15.83:0/3373894701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b01a63b0 con 0x7f84b0100810 2024-04-25T11:41:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84affff640 1 --2- 172.21.15.83:0/3373894701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84b0103180 0x7f84b019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b4f49640 1 --2- 172.21.15.83:0/3373894701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b0100810 0x7f84b019e790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f84a400e9f0 tx=0x7f84a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84adffb640 1 -- 172.21.15.83:0/3373894701 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a400cd80 con 0x7f84b0100810 2024-04-25T11:41:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84adffb640 1 -- 172.21.15.83:0/3373894701 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84a40108e0 con 0x7f84b0100810 2024-04-25T11:41:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84b71ad640 1 -- 172.21.15.83:0/3373894701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b01a66a0 con 0x7f84b0100810 2024-04-25T11:41:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.827+0000 7f84adffb640 1 -- 172.21.15.83:0/3373894701 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a4018980 con 0x7f84b0100810 2024-04-25T11:41:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.828+0000 7f84b71ad640 1 -- 172.21.15.83:0/3373894701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b01a6bf0 con 0x7f84b0100810 2024-04-25T11:41:51.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.829+0000 7f84b71ad640 1 -- 172.21.15.83:0/3373894701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b00fdf40 con 0x7f84b0100810 2024-04-25T11:41:51.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.830+0000 7f84adffb640 1 -- 172.21.15.83:0/3373894701 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f84a4010430 con 0x7f84b0100810 2024-04-25T11:41:51.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.830+0000 7f84adffb640 1 --2- 172.21.15.83:0/3373894701 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84a00a3bb0 0x7f84a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:51.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.831+0000 7f84affff640 1 --2- 172.21.15.83:0/3373894701 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84a00a3bb0 0x7f84a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:51.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.831+0000 7f84adffb640 1 -- 172.21.15.83:0/3373894701 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f84a40d0c80 con 0x7f84b0100810 2024-04-25T11:41:51.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.831+0000 7f84affff640 1 --2- 172.21.15.83:0/3373894701 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84a00a3bb0 0x7f84a00a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f849c006010 tx=0x7f849c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:51.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:51.838+0000 7f84adffb640 1 -- 172.21.15.83:0/3373894701 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84a409d710 con 0x7f84b0100810 2024-04-25T11:41:52.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.026+0000 7f84b71ad640 1 -- 172.21.15.83:0/3373894701 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84b0062b50 con 0x7f84b0100810 2024-04-25T11:41:52.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.029+0000 7f84adffb640 1 -- 172.21.15.83:0/3373894701 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22147 (secure 0 0 0) 0x7f84a409d010 con 0x7f84b0100810 2024-04-25T11:41:52.015 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:52.016 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:41:45.888504+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:41:42.546991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:41:43.764226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:41:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.031+0000 7f84b71ad640 1 -- 172.21.15.83:0/3373894701 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84a00a3bb0 msgr2=0x7f84a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.031+0000 7f84b71ad640 1 --2- 172.21.15.83:0/3373894701 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84a00a3bb0 0x7f84a00a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f849c006010 tx=0x7f849c000bc0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.031+0000 7f84b71ad640 1 -- 172.21.15.83:0/3373894701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b0100810 msgr2=0x7f84b019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.031+0000 7f84b71ad640 1 --2- 172.21.15.83:0/3373894701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b0100810 0x7f84b019e790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f84a400e9f0 tx=0x7f84a400eeb0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.031+0000 7f84b71ad640 1 -- 172.21.15.83:0/3373894701 shutdown_connections 2024-04-25T11:41:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.031+0000 7f84b71ad640 1 --2- 172.21.15.83:0/3373894701 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f84a00a3bb0 0x7f84a00a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.031+0000 7f84b71ad640 1 --2- 172.21.15.83:0/3373894701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84b0105b30 0x7f84b01a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.031+0000 7f84b71ad640 1 --2- 172.21.15.83:0/3373894701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84b0103180 0x7f84b019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.031+0000 7f84b71ad640 1 --2- 172.21.15.83:0/3373894701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b0100810 0x7f84b019e790 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.031+0000 7f84b71ad640 1 -- 172.21.15.83:0/3373894701 >> 172.21.15.83:0/3373894701 conn(0x7f84b00fc450 msgr2=0x7f84b01011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:52.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.031+0000 7f84b71ad640 1 -- 172.21.15.83:0/3373894701 shutdown_connections 2024-04-25T11:41:52.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.031+0000 7f84b71ad640 1 -- 172.21.15.83:0/3373894701 wait complete. 2024-04-25T11:41:52.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:51 smithi094 ceph-mon[33032]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:52.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:51 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] up:active 2024-04-25T11:41:52.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:51 smithi094 ceph-mon[33032]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T11:41:52.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:51 smithi094 ceph-mon[33032]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T11:41:52.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:51 smithi094 ceph-mon[33032]: Cluster is now healthy 2024-04-25T11:41:52.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:51 smithi094 ceph-mon[33032]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-04-25T11:41:52.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:51 smithi094 ceph-mon[33032]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-25T11:41:52.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:51 smithi094 ceph-mon[33032]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-04-25T11:41:52.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:51 smithi094 ceph-mon[33032]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:52.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:51 smithi094 ceph-mon[33032]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:52.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:51 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] up:active 2024-04-25T11:41:52.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:51 smithi094 ceph-mon[33032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T11:41:52.029 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:41:52.030 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:41:52.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:51 smithi111 ceph-mon[32784]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:52.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:51 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] up:active 2024-04-25T11:41:52.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:51 smithi111 ceph-mon[32784]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-25T11:41:52.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:51 smithi111 ceph-mon[32784]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T11:41:52.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:51 smithi111 ceph-mon[32784]: Cluster is now healthy 2024-04-25T11:41:52.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:51 smithi111 ceph-mon[32784]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2024-04-25T11:41:52.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:51 smithi111 ceph-mon[32784]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-25T11:41:52.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:51 smithi111 ceph-mon[32784]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-04-25T11:41:52.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:51 smithi111 ceph-mon[32784]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:52.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:51 smithi111 ceph-mon[32784]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:52.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:51 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] up:active 2024-04-25T11:41:52.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:51 smithi111 ceph-mon[32784]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T11:41:52.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.145+0000 7f4dc3bfe640 1 -- 172.21.15.83:0/3122292502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dbc102a70 msgr2=0x7f4dbc104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.145+0000 7f4dc3bfe640 1 --2- 172.21.15.83:0/3122292502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dbc102a70 0x7f4dbc104e60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009900 tx=0x7f4db802a410 comp rx=0 tx=0).stop 2024-04-25T11:41:52.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.145+0000 7f4dc3bfe640 1 -- 172.21.15.83:0/3122292502 shutdown_connections 2024-04-25T11:41:52.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.145+0000 7f4dc3bfe640 1 --2- 172.21.15.83:0/3122292502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dbc107cd0 0x7f4dbc10a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.145+0000 7f4dc3bfe640 1 --2- 172.21.15.83:0/3122292502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dbc1053a0 0x7f4dbc107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.145+0000 7f4dc3bfe640 1 --2- 172.21.15.83:0/3122292502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dbc102a70 0x7f4dbc104e60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.145+0000 7f4dc3bfe640 1 -- 172.21.15.83:0/3122292502 >> 172.21.15.83:0/3122292502 conn(0x7f4dbc0fc400 msgr2=0x7f4dbc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:52.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.145+0000 7f4dc3bfe640 1 -- 172.21.15.83:0/3122292502 shutdown_connections 2024-04-25T11:41:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.145+0000 7f4dc3bfe640 1 -- 172.21.15.83:0/3122292502 wait complete. 2024-04-25T11:41:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.145+0000 7f4dc3bfe640 1 Processor -- start 2024-04-25T11:41:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.145+0000 7f4dc3bfe640 1 -- start start 2024-04-25T11:41:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.145+0000 7f4dc3bfe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dbc102a70 0x7f4dbc1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.145+0000 7f4dc3bfe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dbc1053a0 0x7f4dbc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.145+0000 7f4dc3bfe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dbc107cd0 0x7f4dbc19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc3bfe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc10db10 con 0x7f4dbc1053a0 2024-04-25T11:41:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc3bfe640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc10d990 con 0x7f4dbc107cd0 2024-04-25T11:41:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc3bfe640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc10dc90 con 0x7f4dbc102a70 2024-04-25T11:41:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc1199640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dbc1053a0 0x7f4dbc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc1199640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dbc1053a0 0x7f4dbc198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43300/0 (socket says 172.21.15.83:43300) 2024-04-25T11:41:52.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc1199640 1 -- 172.21.15.83:0/1967103302 learned_addr learned my addr 172.21.15.83:0/1967103302 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc199a640 1 --2- 172.21.15.83:0/1967103302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dbc102a70 0x7f4dbc1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc219b640 1 --2- 172.21.15.83:0/1967103302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dbc107cd0 0x7f4dbc19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc1199640 1 -- 172.21.15.83:0/1967103302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dbc102a70 msgr2=0x7f4dbc1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc1199640 1 --2- 172.21.15.83:0/1967103302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dbc102a70 0x7f4dbc1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc1199640 1 -- 172.21.15.83:0/1967103302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dbc107cd0 msgr2=0x7f4dbc19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc1199640 1 --2- 172.21.15.83:0/1967103302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dbc107cd0 0x7f4dbc19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc1199640 1 -- 172.21.15.83:0/1967103302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dbc19fde0 con 0x7f4dbc1053a0 2024-04-25T11:41:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc219b640 1 --2- 172.21.15.83:0/1967103302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dbc107cd0 0x7f4dbc19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:41:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc1199640 1 --2- 172.21.15.83:0/1967103302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dbc1053a0 0x7f4dbc198700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00c970 tx=0x7f4dac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4db2ffd640 1 -- 172.21.15.83:0/1967103302 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac014070 con 0x7f4dbc1053a0 2024-04-25T11:41:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4db2ffd640 1 -- 172.21.15.83:0/1967103302 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dac004440 con 0x7f4dbc1053a0 2024-04-25T11:41:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc3bfe640 1 -- 172.21.15.83:0/1967103302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dbc1a00d0 con 0x7f4dbc1053a0 2024-04-25T11:41:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4db2ffd640 1 -- 172.21.15.83:0/1967103302 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac003e50 con 0x7f4dbc1053a0 2024-04-25T11:41:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.146+0000 7f4dc3bfe640 1 -- 172.21.15.83:0/1967103302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dbc1a0620 con 0x7f4dbc1053a0 2024-04-25T11:41:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.147+0000 7f4dc3bfe640 1 -- 172.21.15.83:0/1967103302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dbc0fdef0 con 0x7f4dbc1053a0 2024-04-25T11:41:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.149+0000 7f4db2ffd640 1 -- 172.21.15.83:0/1967103302 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4dac017070 con 0x7f4dbc1053a0 2024-04-25T11:41:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.150+0000 7f4db2ffd640 1 --2- 172.21.15.83:0/1967103302 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4da80a3d50 0x7f4da80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.150+0000 7f4dc199a640 1 --2- 172.21.15.83:0/1967103302 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4da80a3d50 0x7f4da80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.150+0000 7f4db2ffd640 1 -- 172.21.15.83:0/1967103302 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4dac0cfee0 con 0x7f4dbc1053a0 2024-04-25T11:41:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.150+0000 7f4dc199a640 1 --2- 172.21.15.83:0/1967103302 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4da80a3d50 0x7f4da80a6210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4db8000c00 tx=0x7f4db80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:52.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.155+0000 7f4db2ffd640 1 -- 172.21.15.83:0/1967103302 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4dac01e030 con 0x7f4dbc1053a0 2024-04-25T11:41:52.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.335+0000 7f4dc3bfe640 1 -- 172.21.15.83:0/1967103302 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4dbc062390 con 0x7f4da80a3d50 2024-04-25T11:41:52.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.343+0000 7f4db2ffd640 1 -- 172.21.15.83:0/1967103302 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258748 (secure 0 0 0) 0x7f4dbc062390 con 0x7f4da80a3d50 2024-04-25T11:41:52.333 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:52.334 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T11:41:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.346+0000 7f4dc3bfe640 1 -- 172.21.15.83:0/1967103302 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4da80a3d50 msgr2=0x7f4da80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.346+0000 7f4dc3bfe640 1 --2- 172.21.15.83:0/1967103302 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4da80a3d50 0x7f4da80a6210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4db8000c00 tx=0x7f4db80314a0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.346+0000 7f4dc3bfe640 1 -- 172.21.15.83:0/1967103302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dbc1053a0 msgr2=0x7f4dbc198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.346+0000 7f4dc3bfe640 1 --2- 172.21.15.83:0/1967103302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dbc1053a0 0x7f4dbc198700 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00c970 tx=0x7f4dac00ce30 comp rx=0 tx=0).stop 2024-04-25T11:41:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.347+0000 7f4dc3bfe640 1 -- 172.21.15.83:0/1967103302 shutdown_connections 2024-04-25T11:41:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.347+0000 7f4dc3bfe640 1 --2- 172.21.15.83:0/1967103302 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4da80a3d50 0x7f4da80a6210 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.347+0000 7f4dc3bfe640 1 --2- 172.21.15.83:0/1967103302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dbc1053a0 0x7f4dbc198700 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.347+0000 7f4dc3bfe640 1 --2- 172.21.15.83:0/1967103302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4dbc107cd0 0x7f4dbc19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.347+0000 7f4dc3bfe640 1 --2- 172.21.15.83:0/1967103302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dbc102a70 0x7f4dbc1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.347+0000 7f4dc3bfe640 1 -- 172.21.15.83:0/1967103302 >> 172.21.15.83:0/1967103302 conn(0x7f4dbc0fc400 msgr2=0x7f4dbc1033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.347+0000 7f4dc3bfe640 1 -- 172.21.15.83:0/1967103302 shutdown_connections 2024-04-25T11:41:52.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.347+0000 7f4dc3bfe640 1 -- 172.21.15.83:0/1967103302 wait complete. 2024-04-25T11:41:52.344 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-04-25T11:41:51.908302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596513,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":141,"num_read_kb":123,"num_write":327,"num_write_kb":2972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":263,"ondisk_log_size":263,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":282,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374436,"kb_used_data":12780,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567388,"statfs":{"total":1151940427776,"available":1151557005312,"internally_reserved":0,"allocated":13086720,"data_stored":7111019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6145,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":146,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.470259"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388116+0000","last_change":"2024-04-25T11:41:44.907326+0000","last_active":"2024-04-25T11:41:46.388116+0000","last_peered":"2024-04-25T11:41:46.388116+0000","last_clean":"2024-04-25T11:41:46.388116+0000","last_became_active":"2024-04-25T11:41:44.906964+0000","last_became_peered":"2024-04-25T11:41:44.906964+0000","last_unstale":"2024-04-25T11:41:46.388116+0000","last_undegraded":"2024-04-25T11:41:46.388116+0000","last_fullsized":"2024-04-25T11:41:46.388116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:46.656252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387140+0000","last_change":"2024-04-25T11:41:44.907601+0000","last_active":"2024-04-25T11:41:46.387140+0000","last_peered":"2024-04-25T11:41:46.387140+0000","last_clean":"2024-04-25T11:41:46.387140+0000","last_became_active":"2024-04-25T11:41:44.907142+0000","last_became_peered":"2024-04-25T11:41:44.907142+0000","last_unstale":"2024-04-25T11:41:46.387140+0000","last_undegraded":"2024-04-25T11:41:46.387140+0000","last_fullsized":"2024-04-25T11:41:46.387140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:41.901259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.386854+0000","last_change":"2024-04-25T11:41:44.910479+0000","last_active":"2024-04-25T11:41:46.386854+0000","last_peered":"2024-04-25T11:41:46.386854+0000","last_clean":"2024-04-25T11:41:46.386854+0000","last_became_active":"2024-04-25T11:41:44.909711+0000","last_became_peered":"2024-04-25T11:41:44.909711+0000","last_unstale":"2024-04-25T11:41:46.386854+0000","last_undegraded":"2024-04-25T11:41:46.386854+0000","last_fullsized":"2024-04-25T11:41:46.386854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.586793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.892455+0000","last_change":"2024-04-25T11:41:44.909936+0000","last_active":"2024-04-25T11:41:45.892455+0000","last_peered":"2024-04-25T11:41:45.892455+0000","last_clean":"2024-04-25T11:41:45.892455+0000","last_became_active":"2024-04-25T11:41:44.909645+0000","last_became_peered":"2024-04-25T11:41:44.909645+0000","last_unstale":"2024-04-25T11:41:45.892455+0000","last_undegraded":"2024-04-25T11:41:45.892455+0000","last_fullsized":"2024-04-25T11:41:45.892455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:12.518017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.602577+0000","last_change":"2024-04-25T11:41:44.913907+0000","last_active":"2024-04-25T11:41:46.602577+0000","last_peered":"2024-04-25T11:41:46.602577+0000","last_clean":"2024-04-25T11:41:46.602577+0000","last_became_active":"2024-04-25T11:41:44.912593+0000","last_became_peered":"2024-04-25T11:41:44.912593+0000","last_unstale":"2024-04-25T11:41:46.602577+0000","last_undegraded":"2024-04-25T11:41:46.602577+0000","last_fullsized":"2024-04-25T11:41:46.602577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:40.164688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388490+0000","last_change":"2024-04-25T11:41:44.903237+0000","last_active":"2024-04-25T11:41:46.388490+0000","last_peered":"2024-04-25T11:41:46.388490+0000","last_clean":"2024-04-25T11:41:46.388490+0000","last_became_active":"2024-04-25T11:41:44.902881+0000","last_became_peered":"2024-04-25T11:41:44.902881+0000","last_unstale":"2024-04-25T11:41:46.388490+0000","last_undegraded":"2024-04-25T11:41:46.388490+0000","last_fullsized":"2024-04-25T11:41:46.388490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:36.201322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388417+0000","last_change":"2024-04-25T11:41:44.907497+0000","last_active":"2024-04-25T11:41:46.388417+0000","last_peered":"2024-04-25T11:41:46.388417+0000","last_clean":"2024-04-25T11:41:46.388417+0000","last_became_active":"2024-04-25T11:41:44.907117+0000","last_became_peered":"2024-04-25T11:41:44.907117+0000","last_unstale":"2024-04-25T11:41:46.388417+0000","last_undegraded":"2024-04-25T11:41:46.388417+0000","last_fullsized":"2024-04-25T11:41:46.388417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:25.987238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.893905+0000","last_change":"2024-04-25T11:41:44.915499+0000","last_active":"2024-04-25T11:41:45.893905+0000","last_peered":"2024-04-25T11:41:45.893905+0000","last_clean":"2024-04-25T11:41:45.893905+0000","last_became_active":"2024-04-25T11:41:44.915221+0000","last_became_peered":"2024-04-25T11:41:44.915221+0000","last_unstale":"2024-04-25T11:41:45.893905+0000","last_undegraded":"2024-04-25T11:41:45.893905+0000","last_fullsized":"2024-04-25T11:41:45.893905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:36.717428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388735+0000","last_change":"2024-04-25T11:41:44.910599+0000","last_active":"2024-04-25T11:41:46.388735+0000","last_peered":"2024-04-25T11:41:46.388735+0000","last_clean":"2024-04-25T11:41:46.388735+0000","last_became_active":"2024-04-25T11:41:44.908747+0000","last_became_peered":"2024-04-25T11:41:44.908747+0000","last_unstale":"2024-04-25T11:41:46.388735+0000","last_undegraded":"2024-04-25T11:41:46.388735+0000","last_fullsized":"2024-04-25T11:41:46.388735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:50.281168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388662+0000","last_change":"2024-04-25T11:41:44.910692+0000","last_active":"2024-04-25T11:41:46.388662+0000","last_peered":"2024-04-25T11:41:46.388662+0000","last_clean":"2024-04-25T11:41:46.388662+0000","last_became_active":"2024-04-25T11:41:44.910343+0000","last_became_peered":"2024-04-25T11:41:44.910343+0000","last_unstale":"2024-04-25T11:41:46.388662+0000","last_undegraded":"2024-04-25T11:41:46.388662+0000","last_fullsized":"2024-04-25T11:41:46.388662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.945563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387859+0000","last_change":"2024-04-25T11:41:44.910751+0000","last_active":"2024-04-25T11:41:46.387859+0000","last_peered":"2024-04-25T11:41:46.387859+0000","last_clean":"2024-04-25T11:41:46.387859+0000","last_became_active":"2024-04-25T11:41:44.909686+0000","last_became_peered":"2024-04-25T11:41:44.909686+0000","last_unstale":"2024-04-25T11:41:46.387859+0000","last_undegraded":"2024-04-25T11:41:46.387859+0000","last_fullsized":"2024-04-25T11:41:46.387859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:37.216282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.892373+0000","last_change":"2024-04-25T11:41:44.915148+0000","last_active":"2024-04-25T11:41:45.892373+0000","last_peered":"2024-04-25T11:41:45.892373+0000","last_clean":"2024-04-25T11:41:45.892373+0000","last_became_active":"2024-04-25T11:41:44.914772+0000","last_became_peered":"2024-04-25T11:41:44.914772+0000","last_unstale":"2024-04-25T11:41:45.892373+0000","last_undegraded":"2024-04-25T11:41:45.892373+0000","last_fullsized":"2024-04-25T11:41:45.892373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:09.121386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387616+0000","last_change":"2024-04-25T11:41:44.906569+0000","last_active":"2024-04-25T11:41:46.387616+0000","last_peered":"2024-04-25T11:41:46.387616+0000","last_clean":"2024-04-25T11:41:46.387616+0000","last_became_active":"2024-04-25T11:41:44.906259+0000","last_became_peered":"2024-04-25T11:41:44.906259+0000","last_unstale":"2024-04-25T11:41:46.387616+0000","last_undegraded":"2024-04-25T11:41:46.387616+0000","last_fullsized":"2024-04-25T11:41:46.387616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:47.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.386954+0000","last_change":"2024-04-25T11:41:44.909261+0000","last_active":"2024-04-25T11:41:46.386954+0000","last_peered":"2024-04-25T11:41:46.386954+0000","last_clean":"2024-04-25T11:41:46.386954+0000","last_became_active":"2024-04-25T11:41:44.907931+0000","last_became_peered":"2024-04-25T11:41:44.907931+0000","last_unstale":"2024-04-25T11:41:46.386954+0000","last_undegraded":"2024-04-25T11:41:46.386954+0000","last_fullsized":"2024-04-25T11:41:46.386954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:35.754990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.386604+0000","last_change":"2024-04-25T11:41:44.908695+0000","last_active":"2024-04-25T11:41:46.386604+0000","last_peered":"2024-04-25T11:41:46.386604+0000","last_clean":"2024-04-25T11:41:46.386604+0000","last_became_active":"2024-04-25T11:41:44.908374+0000","last_became_peered":"2024-04-25T11:41:44.908374+0000","last_unstale":"2024-04-25T11:41:46.386604+0000","last_undegraded":"2024-04-25T11:41:46.386604+0000","last_fullsized":"2024-04-25T11:41:46.386604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:30.120631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387003+0000","last_change":"2024-04-25T11:41:44.903279+0000","last_active":"2024-04-25T11:41:46.387003+0000","last_peered":"2024-04-25T11:41:46.387003+0000","last_clean":"2024-04-25T11:41:46.387003+0000","last_became_active":"2024-04-25T11:41:44.902958+0000","last_became_peered":"2024-04-25T11:41:44.902958+0000","last_unstale":"2024-04-25T11:41:46.387003+0000","last_undegraded":"2024-04-25T11:41:46.387003+0000","last_fullsized":"2024-04-25T11:41:46.387003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:18.380774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388263+0000","last_change":"2024-04-25T11:41:44.908108+0000","last_active":"2024-04-25T11:41:46.388263+0000","last_peered":"2024-04-25T11:41:46.388263+0000","last_clean":"2024-04-25T11:41:46.388263+0000","last_became_active":"2024-04-25T11:41:44.907787+0000","last_became_peered":"2024-04-25T11:41:44.907787+0000","last_unstale":"2024-04-25T11:41:46.388263+0000","last_undegraded":"2024-04-25T11:41:46.388263+0000","last_fullsized":"2024-04-25T11:41:46.388263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:53.306845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387122+0000","last_change":"2024-04-25T11:41:44.906630+0000","last_active":"2024-04-25T11:41:46.387122+0000","last_peered":"2024-04-25T11:41:46.387122+0000","last_clean":"2024-04-25T11:41:46.387122+0000","last_became_active":"2024-04-25T11:41:44.906321+0000","last_became_peered":"2024-04-25T11:41:44.906321+0000","last_unstale":"2024-04-25T11:41:46.387122+0000","last_undegraded":"2024-04-25T11:41:46.387122+0000","last_fullsized":"2024-04-25T11:41:46.387122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:25.588918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387863+0000","last_change":"2024-04-25T11:41:44.905390+0000","last_active":"2024-04-25T11:41:46.387863+0000","last_peered":"2024-04-25T11:41:46.387863+0000","last_clean":"2024-04-25T11:41:46.387863+0000","last_became_active":"2024-04-25T11:41:44.905033+0000","last_became_peered":"2024-04-25T11:41:44.905033+0000","last_unstale":"2024-04-25T11:41:46.387863+0000","last_undegraded":"2024-04-25T11:41:46.387863+0000","last_fullsized":"2024-04-25T11:41:46.387863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:00.998434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'11","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:49.618681+0000","last_change":"2024-04-25T11:41:44.454718+0000","last_active":"2024-04-25T11:41:49.618681+0000","last_peered":"2024-04-25T11:41:49.618681+0000","last_clean":"2024-04-25T11:41:49.618681+0000","last_became_active":"2024-04-25T11:41:44.454400+0000","last_became_peered":"2024-04-25T11:41:44.454400+0000","last_unstale":"2024-04-25T11:41:49.618681+0000","last_undegraded":"2024-04-25T11:41:49.618681+0000","last_fullsized":"2024-04-25T11:41:49.618681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:05.965988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894254+0000","last_change":"2024-04-25T11:41:44.913512+0000","last_active":"2024-04-25T11:41:45.894254+0000","last_peered":"2024-04-25T11:41:45.894254+0000","last_clean":"2024-04-25T11:41:45.894254+0000","last_became_active":"2024-04-25T11:41:44.913168+0000","last_became_peered":"2024-04-25T11:41:44.913168+0000","last_unstale":"2024-04-25T11:41:45.894254+0000","last_undegraded":"2024-04-25T11:41:45.894254+0000","last_fullsized":"2024-04-25T11:41:45.894254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:25.139747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.429119+0000","last_change":"2024-04-25T11:41:44.451331+0000","last_active":"2024-04-25T11:41:46.429119+0000","last_peered":"2024-04-25T11:41:46.429119+0000","last_clean":"2024-04-25T11:41:46.429119+0000","last_became_active":"2024-04-25T11:41:44.450788+0000","last_became_peered":"2024-04-25T11:41:44.450788+0000","last_unstale":"2024-04-25T11:41:46.429119+0000","last_undegraded":"2024-04-25T11:41:46.429119+0000","last_fullsized":"2024-04-25T11:41:46.429119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:49.011558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387341+0000","last_change":"2024-04-25T11:41:44.908536+0000","last_active":"2024-04-25T11:41:46.387341+0000","last_peered":"2024-04-25T11:41:46.387341+0000","last_clean":"2024-04-25T11:41:46.387341+0000","last_became_active":"2024-04-25T11:41:44.907823+0000","last_became_peered":"2024-04-25T11:41:44.907823+0000","last_unstale":"2024-04-25T11:41:46.387341+0000","last_undegraded":"2024-04-25T11:41:46.387341+0000","last_fullsized":"2024-04-25T11:41:46.387341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:19.898695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.602853+0000","last_change":"2024-04-25T11:41:44.454510+0000","last_active":"2024-04-25T11:41:46.602853+0000","last_peered":"2024-04-25T11:41:46.602853+0000","last_clean":"2024-04-25T11:41:46.602853+0000","last_became_active":"2024-04-25T11:41:44.453452+0000","last_became_peered":"2024-04-25T11:41:44.453452+0000","last_unstale":"2024-04-25T11:41:46.602853+0000","last_undegraded":"2024-04-25T11:41:46.602853+0000","last_fullsized":"2024-04-25T11:41:46.602853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:13.945465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388228+0000","last_change":"2024-04-25T11:41:44.908881+0000","last_active":"2024-04-25T11:41:46.388228+0000","last_peered":"2024-04-25T11:41:46.388228+0000","last_clean":"2024-04-25T11:41:46.388228+0000","last_became_active":"2024-04-25T11:41:44.908493+0000","last_became_peered":"2024-04-25T11:41:44.908493+0000","last_unstale":"2024-04-25T11:41:46.388228+0000","last_undegraded":"2024-04-25T11:41:46.388228+0000","last_fullsized":"2024-04-25T11:41:46.388228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:58.930420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.602575+0000","last_change":"2024-04-25T11:41:44.451870+0000","last_active":"2024-04-25T11:41:46.602575+0000","last_peered":"2024-04-25T11:41:46.602575+0000","last_clean":"2024-04-25T11:41:46.602575+0000","last_became_active":"2024-04-25T11:41:44.451511+0000","last_became_peered":"2024-04-25T11:41:44.451511+0000","last_unstale":"2024-04-25T11:41:46.602575+0000","last_undegraded":"2024-04-25T11:41:46.602575+0000","last_fullsized":"2024-04-25T11:41:46.602575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:26.457919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.895037+0000","last_change":"2024-04-25T11:41:44.910738+0000","last_active":"2024-04-25T11:41:45.895037+0000","last_peered":"2024-04-25T11:41:45.895037+0000","last_clean":"2024-04-25T11:41:45.895037+0000","last_became_active":"2024-04-25T11:41:44.910415+0000","last_became_peered":"2024-04-25T11:41:44.910415+0000","last_unstale":"2024-04-25T11:41:45.895037+0000","last_undegraded":"2024-04-25T11:41:45.895037+0000","last_fullsized":"2024-04-25T11:41:45.895037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:01.333894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.600889+0000","last_change":"2024-04-25T11:41:44.455288+0000","last_active":"2024-04-25T11:41:47.600889+0000","last_peered":"2024-04-25T11:41:47.600889+0000","last_clean":"2024-04-25T11:41:47.600889+0000","last_became_active":"2024-04-25T11:41:44.454965+0000","last_became_peered":"2024-04-25T11:41:44.454965+0000","last_unstale":"2024-04-25T11:41:47.600889+0000","last_undegraded":"2024-04-25T11:41:47.600889+0000","last_fullsized":"2024-04-25T11:41:47.600889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:33.943287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.602512+0000","last_change":"2024-04-25T11:41:44.914274+0000","last_active":"2024-04-25T11:41:46.602512+0000","last_peered":"2024-04-25T11:41:46.602512+0000","last_clean":"2024-04-25T11:41:46.602512+0000","last_became_active":"2024-04-25T11:41:44.913760+0000","last_became_peered":"2024-04-25T11:41:44.913760+0000","last_unstale":"2024-04-25T11:41:46.602512+0000","last_undegraded":"2024-04-25T11:41:46.602512+0000","last_fullsized":"2024-04-25T11:41:46.602512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:57.208895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:48.613484+0000","last_change":"2024-04-25T11:41:44.451846+0000","last_active":"2024-04-25T11:41:48.613484+0000","last_peered":"2024-04-25T11:41:48.613484+0000","last_clean":"2024-04-25T11:41:48.613484+0000","last_became_active":"2024-04-25T11:41:44.451416+0000","last_became_peered":"2024-04-25T11:41:44.451416+0000","last_unstale":"2024-04-25T11:41:48.613484+0000","last_undegraded":"2024-04-25T11:41:48.613484+0000","last_fullsized":"2024-04-25T11:41:48.613484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:57.023883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894981+0000","last_change":"2024-04-25T11:41:44.907761+0000","last_active":"2024-04-25T11:41:45.894981+0000","last_peered":"2024-04-25T11:41:45.894981+0000","last_clean":"2024-04-25T11:41:45.894981+0000","last_became_active":"2024-04-25T11:41:44.906743+0000","last_became_peered":"2024-04-25T11:41:44.906743+0000","last_unstale":"2024-04-25T11:41:45.894981+0000","last_undegraded":"2024-04-25T11:41:45.894981+0000","last_fullsized":"2024-04-25T11:41:45.894981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:43.304985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:48.612944+0000","last_change":"2024-04-25T11:41:44.447415+0000","last_active":"2024-04-25T11:41:48.612944+0000","last_peered":"2024-04-25T11:41:48.612944+0000","last_clean":"2024-04-25T11:41:48.612944+0000","last_became_active":"2024-04-25T11:41:44.447091+0000","last_became_peered":"2024-04-25T11:41:44.447091+0000","last_unstale":"2024-04-25T11:41:48.612944+0000","last_undegraded":"2024-04-25T11:41:48.612944+0000","last_fullsized":"2024-04-25T11:41:48.612944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:27.879115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.893989+0000","last_change":"2024-04-25T11:41:44.913350+0000","last_active":"2024-04-25T11:41:45.893989+0000","last_peered":"2024-04-25T11:41:45.893989+0000","last_clean":"2024-04-25T11:41:45.893989+0000","last_became_active":"2024-04-25T11:41:44.912602+0000","last_became_peered":"2024-04-25T11:41:44.912602+0000","last_unstale":"2024-04-25T11:41:45.893989+0000","last_undegraded":"2024-04-25T11:41:45.893989+0000","last_fullsized":"2024-04-25T11:41:45.893989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:03.099520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894387+0000","last_change":"2024-04-25T11:41:44.451283+0000","last_active":"2024-04-25T11:41:45.894387+0000","last_peered":"2024-04-25T11:41:45.894387+0000","last_clean":"2024-04-25T11:41:45.894387+0000","last_became_active":"2024-04-25T11:41:44.450955+0000","last_became_peered":"2024-04-25T11:41:44.450955+0000","last_unstale":"2024-04-25T11:41:45.894387+0000","last_undegraded":"2024-04-25T11:41:45.894387+0000","last_fullsized":"2024-04-25T11:41:45.894387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:00.912227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387363+0000","last_change":"2024-04-25T11:41:44.904055+0000","last_active":"2024-04-25T11:41:46.387363+0000","last_peered":"2024-04-25T11:41:46.387363+0000","last_clean":"2024-04-25T11:41:46.387363+0000","last_became_active":"2024-04-25T11:41:44.903731+0000","last_became_peered":"2024-04-25T11:41:44.903731+0000","last_unstale":"2024-04-25T11:41:46.387363+0000","last_undegraded":"2024-04-25T11:41:46.387363+0000","last_fullsized":"2024-04-25T11:41:46.387363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:09.912305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.599433+0000","last_change":"2024-04-25T11:41:44.444989+0000","last_active":"2024-04-25T11:41:47.599433+0000","last_peered":"2024-04-25T11:41:47.599433+0000","last_clean":"2024-04-25T11:41:47.599433+0000","last_became_active":"2024-04-25T11:41:44.444543+0000","last_became_peered":"2024-04-25T11:41:44.444543+0000","last_unstale":"2024-04-25T11:41:47.599433+0000","last_undegraded":"2024-04-25T11:41:47.599433+0000","last_fullsized":"2024-04-25T11:41:47.599433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:08.957068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387428+0000","last_change":"2024-04-25T11:41:44.907435+0000","last_active":"2024-04-25T11:41:46.387428+0000","last_peered":"2024-04-25T11:41:46.387428+0000","last_clean":"2024-04-25T11:41:46.387428+0000","last_became_active":"2024-04-25T11:41:44.906972+0000","last_became_peered":"2024-04-25T11:41:44.906972+0000","last_unstale":"2024-04-25T11:41:46.387428+0000","last_undegraded":"2024-04-25T11:41:46.387428+0000","last_fullsized":"2024-04-25T11:41:46.387428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:47.011440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.600108+0000","last_change":"2024-04-25T11:41:44.456804+0000","last_active":"2024-04-25T11:41:47.600108+0000","last_peered":"2024-04-25T11:41:47.600108+0000","last_clean":"2024-04-25T11:41:47.600108+0000","last_became_active":"2024-04-25T11:41:44.456472+0000","last_became_peered":"2024-04-25T11:41:44.456472+0000","last_unstale":"2024-04-25T11:41:47.600108+0000","last_undegraded":"2024-04-25T11:41:47.600108+0000","last_fullsized":"2024-04-25T11:41:47.600108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:49.293772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387317+0000","last_change":"2024-04-25T11:41:44.912003+0000","last_active":"2024-04-25T11:41:46.387317+0000","last_peered":"2024-04-25T11:41:46.387317+0000","last_clean":"2024-04-25T11:41:46.387317+0000","last_became_active":"2024-04-25T11:41:44.909923+0000","last_became_peered":"2024-04-25T11:41:44.909923+0000","last_unstale":"2024-04-25T11:41:46.387317+0000","last_undegraded":"2024-04-25T11:41:46.387317+0000","last_fullsized":"2024-04-25T11:41:46.387317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:56.631924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894971+0000","last_change":"2024-04-25T11:41:44.453860+0000","last_active":"2024-04-25T11:41:45.894971+0000","last_peered":"2024-04-25T11:41:45.894971+0000","last_clean":"2024-04-25T11:41:45.894971+0000","last_became_active":"2024-04-25T11:41:44.453536+0000","last_became_peered":"2024-04-25T11:41:44.453536+0000","last_unstale":"2024-04-25T11:41:45.894971+0000","last_undegraded":"2024-04-25T11:41:45.894971+0000","last_fullsized":"2024-04-25T11:41:45.894971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:56.762115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388444+0000","last_change":"2024-04-25T11:41:44.913443+0000","last_active":"2024-04-25T11:41:46.388444+0000","last_peered":"2024-04-25T11:41:46.388444+0000","last_clean":"2024-04-25T11:41:46.388444+0000","last_became_active":"2024-04-25T11:41:44.911350+0000","last_became_peered":"2024-04-25T11:41:44.911350+0000","last_unstale":"2024-04-25T11:41:46.388444+0000","last_undegraded":"2024-04-25T11:41:46.388444+0000","last_fullsized":"2024-04-25T11:41:46.388444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:58.345587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.598911+0000","last_change":"2024-04-25T11:41:44.452286+0000","last_active":"2024-04-25T11:41:47.598911+0000","last_peered":"2024-04-25T11:41:47.598911+0000","last_clean":"2024-04-25T11:41:47.598911+0000","last_became_active":"2024-04-25T11:41:44.451928+0000","last_became_peered":"2024-04-25T11:41:44.451928+0000","last_unstale":"2024-04-25T11:41:47.598911+0000","last_undegraded":"2024-04-25T11:41:47.598911+0000","last_fullsized":"2024-04-25T11:41:47.598911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:21.407375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.602726+0000","last_change":"2024-04-25T11:41:44.903909+0000","last_active":"2024-04-25T11:41:46.602726+0000","last_peered":"2024-04-25T11:41:46.602726+0000","last_clean":"2024-04-25T11:41:46.602726+0000","last_became_active":"2024-04-25T11:41:44.903620+0000","last_became_peered":"2024-04-25T11:41:44.903620+0000","last_unstale":"2024-04-25T11:41:46.602726+0000","last_undegraded":"2024-04-25T11:41:46.602726+0000","last_fullsized":"2024-04-25T11:41:46.602726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:49.910052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:51.584739+0000","last_change":"2024-04-25T11:41:44.454294+0000","last_active":"2024-04-25T11:41:51.584739+0000","last_peered":"2024-04-25T11:41:51.584739+0000","last_clean":"2024-04-25T11:41:51.584739+0000","last_became_active":"2024-04-25T11:41:44.453829+0000","last_became_peered":"2024-04-25T11:41:44.453829+0000","last_unstale":"2024-04-25T11:41:51.584739+0000","last_undegraded":"2024-04-25T11:41:51.584739+0000","last_fullsized":"2024-04-25T11:41:51.584739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:14.617963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.386726+0000","last_change":"2024-04-25T11:41:44.907453+0000","last_active":"2024-04-25T11:41:46.386726+0000","last_peered":"2024-04-25T11:41:46.386726+0000","last_clean":"2024-04-25T11:41:46.386726+0000","last_became_active":"2024-04-25T11:41:44.907126+0000","last_became_peered":"2024-04-25T11:41:44.907126+0000","last_unstale":"2024-04-25T11:41:46.386726+0000","last_undegraded":"2024-04-25T11:41:46.386726+0000","last_fullsized":"2024-04-25T11:41:46.386726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:25.998686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.599807+0000","last_change":"2024-04-25T11:41:44.452258+0000","last_active":"2024-04-25T11:41:47.599807+0000","last_peered":"2024-04-25T11:41:47.599807+0000","last_clean":"2024-04-25T11:41:47.599807+0000","last_became_active":"2024-04-25T11:41:44.451880+0000","last_became_peered":"2024-04-25T11:41:44.451880+0000","last_unstale":"2024-04-25T11:41:47.599807+0000","last_undegraded":"2024-04-25T11:41:47.599807+0000","last_fullsized":"2024-04-25T11:41:47.599807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:39.463406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387631+0000","last_change":"2024-04-25T11:41:44.913841+0000","last_active":"2024-04-25T11:41:46.387631+0000","last_peered":"2024-04-25T11:41:46.387631+0000","last_clean":"2024-04-25T11:41:46.387631+0000","last_became_active":"2024-04-25T11:41:44.913378+0000","last_became_peered":"2024-04-25T11:41:44.913378+0000","last_unstale":"2024-04-25T11:41:46.387631+0000","last_undegraded":"2024-04-25T11:41:46.387631+0000","last_fullsized":"2024-04-25T11:41:46.387631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:24.732899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:49.617003+0000","last_change":"2024-04-25T11:41:44.450531+0000","last_active":"2024-04-25T11:41:49.617003+0000","last_peered":"2024-04-25T11:41:49.617003+0000","last_clean":"2024-04-25T11:41:49.617003+0000","last_became_active":"2024-04-25T11:41:44.450205+0000","last_became_peered":"2024-04-25T11:41:44.450205+0000","last_unstale":"2024-04-25T11:41:49.617003+0000","last_undegraded":"2024-04-25T11:41:49.617003+0000","last_fullsized":"2024-04-25T11:41:49.617003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:51.797969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.386972+0000","last_change":"2024-04-25T11:41:44.911883+0000","last_active":"2024-04-25T11:41:46.386972+0000","last_peered":"2024-04-25T11:41:46.386972+0000","last_clean":"2024-04-25T11:41:46.386972+0000","last_became_active":"2024-04-25T11:41:44.911362+0000","last_became_peered":"2024-04-25T11:41:44.911362+0000","last_unstale":"2024-04-25T11:41:46.386972+0000","last_undegraded":"2024-04-25T11:41:46.386972+0000","last_fullsized":"2024-04-25T11:41:46.386972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:34.273876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'10","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.599347+0000","last_change":"2024-04-25T11:41:44.455844+0000","last_active":"2024-04-25T11:41:47.599347+0000","last_peered":"2024-04-25T11:41:47.599347+0000","last_clean":"2024-04-25T11:41:47.599347+0000","last_became_active":"2024-04-25T11:41:44.455167+0000","last_became_peered":"2024-04-25T11:41:44.455167+0000","last_unstale":"2024-04-25T11:41:47.599347+0000","last_undegraded":"2024-04-25T11:41:47.599347+0000","last_fullsized":"2024-04-25T11:41:47.599347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.886078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894810+0000","last_change":"2024-04-25T11:41:44.912145+0000","last_active":"2024-04-25T11:41:45.894810+0000","last_peered":"2024-04-25T11:41:45.894810+0000","last_clean":"2024-04-25T11:41:45.894810+0000","last_became_active":"2024-04-25T11:41:44.911113+0000","last_became_peered":"2024-04-25T11:41:44.911113+0000","last_unstale":"2024-04-25T11:41:45.894810+0000","last_undegraded":"2024-04-25T11:41:45.894810+0000","last_fullsized":"2024-04-25T11:41:45.894810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:35.983358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.602818+0000","last_change":"2024-04-25T11:41:44.452371+0000","last_active":"2024-04-25T11:41:46.602818+0000","last_peered":"2024-04-25T11:41:46.602818+0000","last_clean":"2024-04-25T11:41:46.602818+0000","last_became_active":"2024-04-25T11:41:44.451406+0000","last_became_peered":"2024-04-25T11:41:44.451406+0000","last_unstale":"2024-04-25T11:41:46.602818+0000","last_undegraded":"2024-04-25T11:41:46.602818+0000","last_fullsized":"2024-04-25T11:41:46.602818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:00.093835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387947+0000","last_change":"2024-04-25T11:41:44.910999+0000","last_active":"2024-04-25T11:41:46.387947+0000","last_peered":"2024-04-25T11:41:46.387947+0000","last_clean":"2024-04-25T11:41:46.387947+0000","last_became_active":"2024-04-25T11:41:44.909749+0000","last_became_peered":"2024-04-25T11:41:44.909749+0000","last_unstale":"2024-04-25T11:41:46.387947+0000","last_undegraded":"2024-04-25T11:41:46.387947+0000","last_fullsized":"2024-04-25T11:41:46.387947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:44.272145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387031+0000","last_change":"2024-04-25T11:41:44.453243+0000","last_active":"2024-04-25T11:41:46.387031+0000","last_peered":"2024-04-25T11:41:46.387031+0000","last_clean":"2024-04-25T11:41:46.387031+0000","last_became_active":"2024-04-25T11:41:44.452925+0000","last_became_peered":"2024-04-25T11:41:44.452925+0000","last_unstale":"2024-04-25T11:41:46.387031+0000","last_undegraded":"2024-04-25T11:41:46.387031+0000","last_fullsized":"2024-04-25T11:41:46.387031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:57.918332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388274+0000","last_change":"2024-04-25T11:41:44.907350+0000","last_active":"2024-04-25T11:41:46.388274+0000","last_peered":"2024-04-25T11:41:46.388274+0000","last_clean":"2024-04-25T11:41:46.388274+0000","last_became_active":"2024-04-25T11:41:44.907000+0000","last_became_peered":"2024-04-25T11:41:44.907000+0000","last_unstale":"2024-04-25T11:41:46.388274+0000","last_undegraded":"2024-04-25T11:41:46.388274+0000","last_fullsized":"2024-04-25T11:41:46.388274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:08.823550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387290+0000","last_change":"2024-04-25T11:41:44.455318+0000","last_active":"2024-04-25T11:41:46.387290+0000","last_peered":"2024-04-25T11:41:46.387290+0000","last_clean":"2024-04-25T11:41:46.387290+0000","last_became_active":"2024-04-25T11:41:44.454972+0000","last_became_peered":"2024-04-25T11:41:44.454972+0000","last_unstale":"2024-04-25T11:41:46.387290+0000","last_undegraded":"2024-04-25T11:41:46.387290+0000","last_fullsized":"2024-04-25T11:41:46.387290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.255860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388648+0000","last_change":"2024-04-25T11:41:44.912507+0000","last_active":"2024-04-25T11:41:46.388648+0000","last_peered":"2024-04-25T11:41:46.388648+0000","last_clean":"2024-04-25T11:41:46.388648+0000","last_became_active":"2024-04-25T11:41:44.912123+0000","last_became_peered":"2024-04-25T11:41:44.912123+0000","last_unstale":"2024-04-25T11:41:46.388648+0000","last_undegraded":"2024-04-25T11:41:46.388648+0000","last_fullsized":"2024-04-25T11:41:46.388648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:40.659961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894716+0000","last_change":"2024-04-25T11:41:44.913505+0000","last_active":"2024-04-25T11:41:45.894716+0000","last_peered":"2024-04-25T11:41:45.894716+0000","last_clean":"2024-04-25T11:41:45.894716+0000","last_became_active":"2024-04-25T11:41:44.913103+0000","last_became_peered":"2024-04-25T11:41:44.913103+0000","last_unstale":"2024-04-25T11:41:45.894716+0000","last_undegraded":"2024-04-25T11:41:45.894716+0000","last_fullsized":"2024-04-25T11:41:45.894716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:49.414802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:48.613838+0000","last_change":"2024-04-25T11:41:44.450056+0000","last_active":"2024-04-25T11:41:48.613838+0000","last_peered":"2024-04-25T11:41:48.613838+0000","last_clean":"2024-04-25T11:41:48.613838+0000","last_became_active":"2024-04-25T11:41:44.449754+0000","last_became_peered":"2024-04-25T11:41:44.449754+0000","last_unstale":"2024-04-25T11:41:48.613838+0000","last_undegraded":"2024-04-25T11:41:48.613838+0000","last_fullsized":"2024-04-25T11:41:48.613838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:30.059852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894315+0000","last_change":"2024-04-25T11:41:44.915061+0000","last_active":"2024-04-25T11:41:45.894315+0000","last_peered":"2024-04-25T11:41:45.894315+0000","last_clean":"2024-04-25T11:41:45.894315+0000","last_became_active":"2024-04-25T11:41:44.914799+0000","last_became_peered":"2024-04-25T11:41:44.914799+0000","last_unstale":"2024-04-25T11:41:45.894315+0000","last_undegraded":"2024-04-25T11:41:45.894315+0000","last_fullsized":"2024-04-25T11:41:45.894315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:22.820539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:49.616317+0000","last_change":"2024-04-25T11:41:44.449248+0000","last_active":"2024-04-25T11:41:49.616317+0000","last_peered":"2024-04-25T11:41:49.616317+0000","last_clean":"2024-04-25T11:41:49.616317+0000","last_became_active":"2024-04-25T11:41:44.448898+0000","last_became_peered":"2024-04-25T11:41:44.448898+0000","last_unstale":"2024-04-25T11:41:49.616317+0000","last_undegraded":"2024-04-25T11:41:49.616317+0000","last_fullsized":"2024-04-25T11:41:49.616317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:19.574270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387095+0000","last_change":"2024-04-25T11:40:36.892561+0000","last_active":"2024-04-25T11:41:46.387095+0000","last_peered":"2024-04-25T11:41:46.387095+0000","last_clean":"2024-04-25T11:41:46.387095+0000","last_became_active":"2024-04-25T11:40:36.892126+0000","last_became_peered":"2024-04-25T11:40:36.892126+0000","last_unstale":"2024-04-25T11:41:46.387095+0000","last_undegraded":"2024-04-25T11:41:46.387095+0000","last_fullsized":"2024-04-25T11:41:46.387095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:37:16.666579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:37:16.666579+0000","last_clean_scrub_stamp":"2024-04-25T11:37:16.666579+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:44.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.386972+0000","last_change":"2024-04-25T11:41:44.908446+0000","last_active":"2024-04-25T11:41:46.386972+0000","last_peered":"2024-04-25T11:41:46.386972+0000","last_clean":"2024-04-25T11:41:46.386972+0000","last_became_active":"2024-04-25T11:41:44.908148+0000","last_became_peered":"2024-04-25T11:41:44.908148+0000","last_unstale":"2024-04-25T11:41:46.386972+0000","last_undegraded":"2024-04-25T11:41:46.386972+0000","last_fullsized":"2024-04-25T11:41:46.386972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:09.598976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:48.612514+0000","last_change":"2024-04-25T11:41:44.454815+0000","last_active":"2024-04-25T11:41:48.612514+0000","last_peered":"2024-04-25T11:41:48.612514+0000","last_clean":"2024-04-25T11:41:48.612514+0000","last_became_active":"2024-04-25T11:41:44.454521+0000","last_became_peered":"2024-04-25T11:41:44.454521+0000","last_unstale":"2024-04-25T11:41:48.612514+0000","last_undegraded":"2024-04-25T11:41:48.612514+0000","last_fullsized":"2024-04-25T11:41:48.612514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:45.932329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.599233+0000","last_change":"2024-04-25T11:41:44.456641+0000","last_active":"2024-04-25T11:41:47.599233+0000","last_peered":"2024-04-25T11:41:47.599233+0000","last_clean":"2024-04-25T11:41:47.599233+0000","last_became_active":"2024-04-25T11:41:44.456133+0000","last_became_peered":"2024-04-25T11:41:44.456133+0000","last_unstale":"2024-04-25T11:41:47.599233+0000","last_undegraded":"2024-04-25T11:41:47.599233+0000","last_fullsized":"2024-04-25T11:41:47.599233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:47.746160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387484+0000","last_change":"2024-04-25T11:41:44.897512+0000","last_active":"2024-04-25T11:41:46.387484+0000","last_peered":"2024-04-25T11:41:46.387484+0000","last_clean":"2024-04-25T11:41:46.387484+0000","last_became_active":"2024-04-25T11:41:44.897184+0000","last_became_peered":"2024-04-25T11:41:44.897184+0000","last_unstale":"2024-04-25T11:41:46.387484+0000","last_undegraded":"2024-04-25T11:41:46.387484+0000","last_fullsized":"2024-04-25T11:41:46.387484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:53.703825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:49.619010+0000","last_change":"2024-04-25T11:41:44.458318+0000","last_active":"2024-04-25T11:41:49.619010+0000","last_peered":"2024-04-25T11:41:49.619010+0000","last_clean":"2024-04-25T11:41:49.619010+0000","last_became_active":"2024-04-25T11:41:44.458001+0000","last_became_peered":"2024-04-25T11:41:44.458001+0000","last_unstale":"2024-04-25T11:41:49.619010+0000","last_undegraded":"2024-04-25T11:41:49.619010+0000","last_fullsized":"2024-04-25T11:41:49.619010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:17.100621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894447+0000","last_change":"2024-04-25T11:41:44.905912+0000","last_active":"2024-04-25T11:41:45.894447+0000","last_peered":"2024-04-25T11:41:45.894447+0000","last_clean":"2024-04-25T11:41:45.894447+0000","last_became_active":"2024-04-25T11:41:44.905606+0000","last_became_peered":"2024-04-25T11:41:44.905606+0000","last_unstale":"2024-04-25T11:41:45.894447+0000","last_undegraded":"2024-04-25T11:41:45.894447+0000","last_fullsized":"2024-04-25T11:41:45.894447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:47.609884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.598152+0000","last_change":"2024-04-25T11:41:44.460492+0000","last_active":"2024-04-25T11:41:47.598152+0000","last_peered":"2024-04-25T11:41:47.598152+0000","last_clean":"2024-04-25T11:41:47.598152+0000","last_became_active":"2024-04-25T11:41:44.460199+0000","last_became_peered":"2024-04-25T11:41:44.460199+0000","last_unstale":"2024-04-25T11:41:47.598152+0000","last_undegraded":"2024-04-25T11:41:47.598152+0000","last_fullsized":"2024-04-25T11:41:47.598152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:18.756779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387236+0000","last_change":"2024-04-25T11:41:44.909438+0000","last_active":"2024-04-25T11:41:46.387236+0000","last_peered":"2024-04-25T11:41:46.387236+0000","last_clean":"2024-04-25T11:41:46.387236+0000","last_became_active":"2024-04-25T11:41:44.909146+0000","last_became_peered":"2024-04-25T11:41:44.909146+0000","last_unstale":"2024-04-25T11:41:46.387236+0000","last_undegraded":"2024-04-25T11:41:46.387236+0000","last_fullsized":"2024-04-25T11:41:46.387236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:15.949652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.598441+0000","last_change":"2024-04-25T11:41:44.453169+0000","last_active":"2024-04-25T11:41:47.598441+0000","last_peered":"2024-04-25T11:41:47.598441+0000","last_clean":"2024-04-25T11:41:47.598441+0000","last_became_active":"2024-04-25T11:41:44.452762+0000","last_became_peered":"2024-04-25T11:41:44.452762+0000","last_unstale":"2024-04-25T11:41:47.598441+0000","last_undegraded":"2024-04-25T11:41:47.598441+0000","last_fullsized":"2024-04-25T11:41:47.598441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:13.084197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.602420+0000","last_change":"2024-04-25T11:41:44.910810+0000","last_active":"2024-04-25T11:41:46.602420+0000","last_peered":"2024-04-25T11:41:46.602420+0000","last_clean":"2024-04-25T11:41:46.602420+0000","last_became_active":"2024-04-25T11:41:44.910488+0000","last_became_peered":"2024-04-25T11:41:44.910488+0000","last_unstale":"2024-04-25T11:41:46.602420+0000","last_undegraded":"2024-04-25T11:41:46.602420+0000","last_fullsized":"2024-04-25T11:41:46.602420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:36.971289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:50.622008+0000","last_change":"2024-04-25T11:41:44.455402+0000","last_active":"2024-04-25T11:41:50.622008+0000","last_peered":"2024-04-25T11:41:50.622008+0000","last_clean":"2024-04-25T11:41:50.622008+0000","last_became_active":"2024-04-25T11:41:44.455057+0000","last_became_peered":"2024-04-25T11:41:44.455057+0000","last_unstale":"2024-04-25T11:41:50.622008+0000","last_undegraded":"2024-04-25T11:41:50.622008+0000","last_fullsized":"2024-04-25T11:41:50.622008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:32.156664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387346+0000","last_change":"2024-04-25T11:41:44.903907+0000","last_active":"2024-04-25T11:41:46.387346+0000","last_peered":"2024-04-25T11:41:46.387346+0000","last_clean":"2024-04-25T11:41:46.387346+0000","last_became_active":"2024-04-25T11:41:44.903510+0000","last_became_peered":"2024-04-25T11:41:44.903510+0000","last_unstale":"2024-04-25T11:41:46.387346+0000","last_undegraded":"2024-04-25T11:41:46.387346+0000","last_fullsized":"2024-04-25T11:41:46.387346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:33.183412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:49.618241+0000","last_change":"2024-04-25T11:41:44.454049+0000","last_active":"2024-04-25T11:41:49.618241+0000","last_peered":"2024-04-25T11:41:49.618241+0000","last_clean":"2024-04-25T11:41:49.618241+0000","last_became_active":"2024-04-25T11:41:44.453652+0000","last_became_peered":"2024-04-25T11:41:44.453652+0000","last_unstale":"2024-04-25T11:41:49.618241+0000","last_undegraded":"2024-04-25T11:41:49.618241+0000","last_fullsized":"2024-04-25T11:41:49.618241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:17.239715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894156+0000","last_change":"2024-04-25T11:41:44.915005+0000","last_active":"2024-04-25T11:41:45.894156+0000","last_peered":"2024-04-25T11:41:45.894156+0000","last_clean":"2024-04-25T11:41:45.894156+0000","last_became_active":"2024-04-25T11:41:44.914663+0000","last_became_peered":"2024-04-25T11:41:44.914663+0000","last_unstale":"2024-04-25T11:41:45.894156+0000","last_undegraded":"2024-04-25T11:41:45.894156+0000","last_fullsized":"2024-04-25T11:41:45.894156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:49.390846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.600185+0000","last_change":"2024-04-25T11:41:44.454194+0000","last_active":"2024-04-25T11:41:47.600185+0000","last_peered":"2024-04-25T11:41:47.600185+0000","last_clean":"2024-04-25T11:41:47.600185+0000","last_became_active":"2024-04-25T11:41:44.453280+0000","last_became_peered":"2024-04-25T11:41:44.453280+0000","last_unstale":"2024-04-25T11:41:47.600185+0000","last_undegraded":"2024-04-25T11:41:47.600185+0000","last_fullsized":"2024-04-25T11:41:47.600185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:45.368132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387371+0000","last_change":"2024-04-25T11:41:44.907951+0000","last_active":"2024-04-25T11:41:46.387371+0000","last_peered":"2024-04-25T11:41:46.387371+0000","last_clean":"2024-04-25T11:41:46.387371+0000","last_became_active":"2024-04-25T11:41:44.907554+0000","last_became_peered":"2024-04-25T11:41:44.907554+0000","last_unstale":"2024-04-25T11:41:46.387371+0000","last_undegraded":"2024-04-25T11:41:46.387371+0000","last_fullsized":"2024-04-25T11:41:46.387371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:33.312265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:51.588428+0000","last_change":"2024-04-25T11:41:44.458803+0000","last_active":"2024-04-25T11:41:51.588428+0000","last_peered":"2024-04-25T11:41:51.588428+0000","last_clean":"2024-04-25T11:41:51.588428+0000","last_became_active":"2024-04-25T11:41:44.458307+0000","last_became_peered":"2024-04-25T11:41:44.458307+0000","last_unstale":"2024-04-25T11:41:51.588428+0000","last_undegraded":"2024-04-25T11:41:51.588428+0000","last_fullsized":"2024-04-25T11:41:51.588428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:26.186429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387897+0000","last_change":"2024-04-25T11:41:44.909600+0000","last_active":"2024-04-25T11:41:46.387897+0000","last_peered":"2024-04-25T11:41:46.387897+0000","last_clean":"2024-04-25T11:41:46.387897+0000","last_became_active":"2024-04-25T11:41:44.909083+0000","last_became_peered":"2024-04-25T11:41:44.909083+0000","last_unstale":"2024-04-25T11:41:46.387897+0000","last_undegraded":"2024-04-25T11:41:46.387897+0000","last_fullsized":"2024-04-25T11:41:46.387897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:53.854631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894690+0000","last_change":"2024-04-25T11:41:44.451466+0000","last_active":"2024-04-25T11:41:45.894690+0000","last_peered":"2024-04-25T11:41:45.894690+0000","last_clean":"2024-04-25T11:41:45.894690+0000","last_became_active":"2024-04-25T11:41:44.450866+0000","last_became_peered":"2024-04-25T11:41:44.450866+0000","last_unstale":"2024-04-25T11:41:45.894690+0000","last_undegraded":"2024-04-25T11:41:45.894690+0000","last_fullsized":"2024-04-25T11:41:45.894690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:03.715468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387474+0000","last_change":"2024-04-25T11:41:44.909789+0000","last_active":"2024-04-25T11:41:46.387474+0000","last_peered":"2024-04-25T11:41:46.387474+0000","last_clean":"2024-04-25T11:41:46.387474+0000","last_became_active":"2024-04-25T11:41:44.909490+0000","last_became_peered":"2024-04-25T11:41:44.909490+0000","last_unstale":"2024-04-25T11:41:46.387474+0000","last_undegraded":"2024-04-25T11:41:46.387474+0000","last_fullsized":"2024-04-25T11:41:46.387474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:50.543847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387146+0000","last_change":"2024-04-25T11:41:44.449371+0000","last_active":"2024-04-25T11:41:46.387146+0000","last_peered":"2024-04-25T11:41:46.387146+0000","last_clean":"2024-04-25T11:41:46.387146+0000","last_became_active":"2024-04-25T11:41:44.448927+0000","last_became_peered":"2024-04-25T11:41:44.448927+0000","last_unstale":"2024-04-25T11:41:46.387146+0000","last_undegraded":"2024-04-25T11:41:46.387146+0000","last_fullsized":"2024-04-25T11:41:46.387146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:49.581010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388120+0000","last_change":"2024-04-25T11:41:44.906540+0000","last_active":"2024-04-25T11:41:46.388120+0000","last_peered":"2024-04-25T11:41:46.388120+0000","last_clean":"2024-04-25T11:41:46.388120+0000","last_became_active":"2024-04-25T11:41:44.905918+0000","last_became_peered":"2024-04-25T11:41:44.905918+0000","last_unstale":"2024-04-25T11:41:46.388120+0000","last_undegraded":"2024-04-25T11:41:46.388120+0000","last_fullsized":"2024-04-25T11:41:46.388120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:38.350461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387314+0000","last_change":"2024-04-25T11:41:44.904656+0000","last_active":"2024-04-25T11:41:46.387314+0000","last_peered":"2024-04-25T11:41:46.387314+0000","last_clean":"2024-04-25T11:41:46.387314+0000","last_became_active":"2024-04-25T11:41:44.904322+0000","last_became_peered":"2024-04-25T11:41:44.904322+0000","last_unstale":"2024-04-25T11:41:46.387314+0000","last_undegraded":"2024-04-25T11:41:46.387314+0000","last_fullsized":"2024-04-25T11:41:46.387314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:07.762816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388192+0000","last_change":"2024-04-25T11:41:44.911662+0000","last_active":"2024-04-25T11:41:46.388192+0000","last_peered":"2024-04-25T11:41:46.388192+0000","last_clean":"2024-04-25T11:41:46.388192+0000","last_became_active":"2024-04-25T11:41:44.911295+0000","last_became_peered":"2024-04-25T11:41:44.911295+0000","last_unstale":"2024-04-25T11:41:46.388192+0000","last_undegraded":"2024-04-25T11:41:46.388192+0000","last_fullsized":"2024-04-25T11:41:46.388192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:12.988118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387426+0000","last_change":"2024-04-25T11:41:44.906303+0000","last_active":"2024-04-25T11:41:46.387426+0000","last_peered":"2024-04-25T11:41:46.387426+0000","last_clean":"2024-04-25T11:41:46.387426+0000","last_became_active":"2024-04-25T11:41:44.905951+0000","last_became_peered":"2024-04-25T11:41:44.905951+0000","last_unstale":"2024-04-25T11:41:46.387426+0000","last_undegraded":"2024-04-25T11:41:46.387426+0000","last_fullsized":"2024-04-25T11:41:46.387426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:49.887856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387090+0000","last_change":"2024-04-25T11:41:44.910457+0000","last_active":"2024-04-25T11:41:46.387090+0000","last_peered":"2024-04-25T11:41:46.387090+0000","last_clean":"2024-04-25T11:41:46.387090+0000","last_became_active":"2024-04-25T11:41:44.909089+0000","last_became_peered":"2024-04-25T11:41:44.909089+0000","last_unstale":"2024-04-25T11:41:46.387090+0000","last_undegraded":"2024-04-25T11:41:46.387090+0000","last_fullsized":"2024-04-25T11:41:46.387090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:25.222918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387530+0000","last_change":"2024-04-25T11:41:44.909391+0000","last_active":"2024-04-25T11:41:46.387530+0000","last_peered":"2024-04-25T11:41:46.387530+0000","last_clean":"2024-04-25T11:41:46.387530+0000","last_became_active":"2024-04-25T11:41:44.909088+0000","last_became_peered":"2024-04-25T11:41:44.909088+0000","last_unstale":"2024-04-25T11:41:46.387530+0000","last_undegraded":"2024-04-25T11:41:46.387530+0000","last_fullsized":"2024-04-25T11:41:46.387530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:38.550716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388028+0000","last_change":"2024-04-25T11:41:44.904351+0000","last_active":"2024-04-25T11:41:46.388028+0000","last_peered":"2024-04-25T11:41:46.388028+0000","last_clean":"2024-04-25T11:41:46.388028+0000","last_became_active":"2024-04-25T11:41:44.904015+0000","last_became_peered":"2024-04-25T11:41:44.904015+0000","last_unstale":"2024-04-25T11:41:46.388028+0000","last_undegraded":"2024-04-25T11:41:46.388028+0000","last_fullsized":"2024-04-25T11:41:46.388028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:18.382370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388125+0000","last_change":"2024-04-25T11:41:44.901469+0000","last_active":"2024-04-25T11:41:46.388125+0000","last_peered":"2024-04-25T11:41:46.388125+0000","last_clean":"2024-04-25T11:41:46.388125+0000","last_became_active":"2024-04-25T11:41:44.900943+0000","last_became_peered":"2024-04-25T11:41:44.900943+0000","last_unstale":"2024-04-25T11:41:46.388125+0000","last_undegraded":"2024-04-25T11:41:46.388125+0000","last_fullsized":"2024-04-25T11:41:46.388125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:33.712132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894089+0000","last_change":"2024-04-25T11:41:44.912267+0000","last_active":"2024-04-25T11:41:45.894089+0000","last_peered":"2024-04-25T11:41:45.894089+0000","last_clean":"2024-04-25T11:41:45.894089+0000","last_became_active":"2024-04-25T11:41:44.911968+0000","last_became_peered":"2024-04-25T11:41:44.911968+0000","last_unstale":"2024-04-25T11:41:45.894089+0000","last_undegraded":"2024-04-25T11:41:45.894089+0000","last_fullsized":"2024-04-25T11:41:45.894089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:27.956046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894613+0000","last_change":"2024-04-25T11:41:44.907540+0000","last_active":"2024-04-25T11:41:45.894613+0000","last_peered":"2024-04-25T11:41:45.894613+0000","last_clean":"2024-04-25T11:41:45.894613+0000","last_became_active":"2024-04-25T11:41:44.907229+0000","last_became_peered":"2024-04-25T11:41:44.907229+0000","last_unstale":"2024-04-25T11:41:45.894613+0000","last_undegraded":"2024-04-25T11:41:45.894613+0000","last_fullsized":"2024-04-25T11:41:45.894613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:49.123196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.893930+0000","last_change":"2024-04-25T11:41:44.912354+0000","last_active":"2024-04-25T11:41:45.893930+0000","last_peered":"2024-04-25T11:41:45.893930+0000","last_clean":"2024-04-25T11:41:45.893930+0000","last_became_active":"2024-04-25T11:41:44.912028+0000","last_became_peered":"2024-04-25T11:41:44.912028+0000","last_unstale":"2024-04-25T11:41:45.893930+0000","last_undegraded":"2024-04-25T11:41:45.893930+0000","last_fullsized":"2024-04-25T11:41:45.893930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:50.824647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387845+0000","last_change":"2024-04-25T11:41:44.907346+0000","last_active":"2024-04-25T11:41:46.387845+0000","last_peered":"2024-04-25T11:41:46.387845+0000","last_clean":"2024-04-25T11:41:46.387845+0000","last_became_active":"2024-04-25T11:41:44.907027+0000","last_became_peered":"2024-04-25T11:41:44.907027+0000","last_unstale":"2024-04-25T11:41:46.387845+0000","last_undegraded":"2024-04-25T11:41:46.387845+0000","last_fullsized":"2024-04-25T11:41:46.387845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:01.913218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387532+0000","last_change":"2024-04-25T11:41:44.902606+0000","last_active":"2024-04-25T11:41:46.387532+0000","last_peered":"2024-04-25T11:41:46.387532+0000","last_clean":"2024-04-25T11:41:46.387532+0000","last_became_active":"2024-04-25T11:41:44.902306+0000","last_became_peered":"2024-04-25T11:41:44.902306+0000","last_unstale":"2024-04-25T11:41:46.387532+0000","last_undegraded":"2024-04-25T11:41:46.387532+0000","last_fullsized":"2024-04-25T11:41:46.387532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:24.214013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387894+0000","last_change":"2024-04-25T11:41:44.909653+0000","last_active":"2024-04-25T11:41:46.387894+0000","last_peered":"2024-04-25T11:41:46.387894+0000","last_clean":"2024-04-25T11:41:46.387894+0000","last_became_active":"2024-04-25T11:41:44.909256+0000","last_became_peered":"2024-04-25T11:41:44.909256+0000","last_unstale":"2024-04-25T11:41:46.387894+0000","last_undegraded":"2024-04-25T11:41:46.387894+0000","last_fullsized":"2024-04-25T11:41:46.387894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:30.310432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6145,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":146,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":299008,"data_stored":53707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350988,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":442151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36852,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708300,"statfs":{"total":95995035648,"available":95957299200,"internally_reserved":0,"allocated":1552384,"data_stored":1035087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":442969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037787,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":442755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299425,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36832,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708320,"statfs":{"total":95995035648,"available":95957319680,"internally_reserved":0,"allocated":1531904,"data_stored":1034311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462950,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":925696,"data_stored":442241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":443763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":442845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":465409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":954368,"data_stored":442698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888112,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":442795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724586,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1544192,"data_stored":1033995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":40556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:41:52.348 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:41:52.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 -- 172.21.15.83:0/1099579423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc101fe0 msgr2=0x7f44bc102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 --2- 172.21.15.83:0/1099579423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc101fe0 0x7f44bc102460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009900 tx=0x7f44ac02a410 comp rx=0 tx=0).stop 2024-04-25T11:41:52.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 -- 172.21.15.83:0/1099579423 shutdown_connections 2024-04-25T11:41:52.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 --2- 172.21.15.83:0/1099579423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc101fe0 0x7f44bc102460 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 --2- 172.21.15.83:0/1099579423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44bc1029a0 0x7f44bc1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 --2- 172.21.15.83:0/1099579423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44bc100e90 0x7f44bc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 -- 172.21.15.83:0/1099579423 >> 172.21.15.83:0/1099579423 conn(0x7f44bc0fc400 msgr2=0x7f44bc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 -- 172.21.15.83:0/1099579423 shutdown_connections 2024-04-25T11:41:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 -- 172.21.15.83:0/1099579423 wait complete. 2024-04-25T11:41:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 Processor -- start 2024-04-25T11:41:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 -- start start 2024-04-25T11:41:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc100e90 0x7f44bc069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44bc101fe0 0x7f44bc19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44bc1029a0 0x7f44bc19f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc10c9d0 con 0x7f44bc100e90 2024-04-25T11:41:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc10c850 con 0x7f44bc1029a0 2024-04-25T11:41:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.499+0000 7f44c34cf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc10cb50 con 0x7f44bc101fe0 2024-04-25T11:41:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44c126b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc100e90 0x7f44bc069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44c126b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc100e90 0x7f44bc069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43316/0 (socket says 172.21.15.83:43316) 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44c126b640 1 -- 172.21.15.83:0/2994656992 learned_addr learned my addr 172.21.15.83:0/2994656992 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44c1a6c640 1 --2- 172.21.15.83:0/2994656992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44bc1029a0 0x7f44bc19f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44c0a6a640 1 --2- 172.21.15.83:0/2994656992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44bc101fe0 0x7f44bc19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44c126b640 1 -- 172.21.15.83:0/2994656992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44bc101fe0 msgr2=0x7f44bc19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44c126b640 1 --2- 172.21.15.83:0/2994656992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44bc101fe0 0x7f44bc19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44c126b640 1 -- 172.21.15.83:0/2994656992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44bc1029a0 msgr2=0x7f44bc19f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44c126b640 1 --2- 172.21.15.83:0/2994656992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44bc1029a0 0x7f44bc19f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44c126b640 1 -- 172.21.15.83:0/2994656992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44bc19fe40 con 0x7f44bc100e90 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44c126b640 1 --2- 172.21.15.83:0/2994656992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc100e90 0x7f44bc069c50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f44b800d950 tx=0x7f44b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44b27fc640 1 -- 172.21.15.83:0/2994656992 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8017070 con 0x7f44bc100e90 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44b27fc640 1 -- 172.21.15.83:0/2994656992 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b8009c30 con 0x7f44bc100e90 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44c34cf640 1 -- 172.21.15.83:0/2994656992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44bc1a00d0 con 0x7f44bc100e90 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44b27fc640 1 -- 172.21.15.83:0/2994656992 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b800f620 con 0x7f44bc100e90 2024-04-25T11:41:52.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.500+0000 7f44c34cf640 1 -- 172.21.15.83:0/2994656992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44bc1a05f0 con 0x7f44bc100e90 2024-04-25T11:41:52.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.501+0000 7f44c34cf640 1 -- 172.21.15.83:0/2994656992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44bc1082c0 con 0x7f44bc100e90 2024-04-25T11:41:52.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.502+0000 7f44b27fc640 1 -- 172.21.15.83:0/2994656992 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f44b8010460 con 0x7f44bc100e90 2024-04-25T11:41:52.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.503+0000 7f44b27fc640 1 --2- 172.21.15.83:0/2994656992 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f44a80a3bb0 0x7f44a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:52.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.503+0000 7f44c0a6a640 1 --2- 172.21.15.83:0/2994656992 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f44a80a3bb0 0x7f44a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:52.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.503+0000 7f44b27fc640 1 -- 172.21.15.83:0/2994656992 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f44b80cfee0 con 0x7f44bc100e90 2024-04-25T11:41:52.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.504+0000 7f44c0a6a640 1 --2- 172.21.15.83:0/2994656992 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f44a80a3bb0 0x7f44a80a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f44ac000c00 tx=0x7f44ac0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:52.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.508+0000 7f44b27fc640 1 -- 172.21.15.83:0/2994656992 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44b809ca10 con 0x7f44bc100e90 2024-04-25T11:41:52.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.630+0000 7f44b27fc640 1 -- 172.21.15.83:0/2994656992 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f44b8098980 con 0x7f44bc100e90 2024-04-25T11:41:52.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.699+0000 7f44c34cf640 1 -- 172.21.15.83:0/2994656992 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44bc062390 con 0x7f44a80a3bb0 2024-04-25T11:41:52.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.706+0000 7f44b27fc640 1 -- 172.21.15.83:0/2994656992 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258748 (secure 0 0 0) 0x7f44bc062390 con 0x7f44a80a3bb0 2024-04-25T11:41:52.695 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:52.696 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T11:41:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.709+0000 7f44c34cf640 1 -- 172.21.15.83:0/2994656992 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f44a80a3bb0 msgr2=0x7f44a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.709+0000 7f44c34cf640 1 --2- 172.21.15.83:0/2994656992 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f44a80a3bb0 0x7f44a80a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f44ac000c00 tx=0x7f44ac0314a0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.709+0000 7f44c34cf640 1 -- 172.21.15.83:0/2994656992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc100e90 msgr2=0x7f44bc069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.709+0000 7f44c34cf640 1 --2- 172.21.15.83:0/2994656992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc100e90 0x7f44bc069c50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f44b800d950 tx=0x7f44b800de10 comp rx=0 tx=0).stop 2024-04-25T11:41:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.709+0000 7f44c34cf640 1 -- 172.21.15.83:0/2994656992 shutdown_connections 2024-04-25T11:41:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.709+0000 7f44c34cf640 1 --2- 172.21.15.83:0/2994656992 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f44a80a3bb0 0x7f44a80a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.709+0000 7f44c34cf640 1 --2- 172.21.15.83:0/2994656992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44bc1029a0 0x7f44bc19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.709+0000 7f44c34cf640 1 --2- 172.21.15.83:0/2994656992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44bc101fe0 0x7f44bc19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.709+0000 7f44c34cf640 1 --2- 172.21.15.83:0/2994656992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc100e90 0x7f44bc069c50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.710+0000 7f44c34cf640 1 -- 172.21.15.83:0/2994656992 >> 172.21.15.83:0/2994656992 conn(0x7f44bc0fc400 msgr2=0x7f44bc105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.710+0000 7f44c34cf640 1 -- 172.21.15.83:0/2994656992 shutdown_connections 2024-04-25T11:41:52.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:52.710+0000 7f44c34cf640 1 -- 172.21.15.83:0/2994656992 wait complete. 2024-04-25T11:41:52.707 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-04-25T11:41:51.908302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596513,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":141,"num_read_kb":123,"num_write":327,"num_write_kb":2972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":263,"ondisk_log_size":263,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":282,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374436,"kb_used_data":12780,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567388,"statfs":{"total":1151940427776,"available":1151557005312,"internally_reserved":0,"allocated":13086720,"data_stored":7111019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6145,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":146,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.470259"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388116+0000","last_change":"2024-04-25T11:41:44.907326+0000","last_active":"2024-04-25T11:41:46.388116+0000","last_peered":"2024-04-25T11:41:46.388116+0000","last_clean":"2024-04-25T11:41:46.388116+0000","last_became_active":"2024-04-25T11:41:44.906964+0000","last_became_peered":"2024-04-25T11:41:44.906964+0000","last_unstale":"2024-04-25T11:41:46.388116+0000","last_undegraded":"2024-04-25T11:41:46.388116+0000","last_fullsized":"2024-04-25T11:41:46.388116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:46.656252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387140+0000","last_change":"2024-04-25T11:41:44.907601+0000","last_active":"2024-04-25T11:41:46.387140+0000","last_peered":"2024-04-25T11:41:46.387140+0000","last_clean":"2024-04-25T11:41:46.387140+0000","last_became_active":"2024-04-25T11:41:44.907142+0000","last_became_peered":"2024-04-25T11:41:44.907142+0000","last_unstale":"2024-04-25T11:41:46.387140+0000","last_undegraded":"2024-04-25T11:41:46.387140+0000","last_fullsized":"2024-04-25T11:41:46.387140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:41.901259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.386854+0000","last_change":"2024-04-25T11:41:44.910479+0000","last_active":"2024-04-25T11:41:46.386854+0000","last_peered":"2024-04-25T11:41:46.386854+0000","last_clean":"2024-04-25T11:41:46.386854+0000","last_became_active":"2024-04-25T11:41:44.909711+0000","last_became_peered":"2024-04-25T11:41:44.909711+0000","last_unstale":"2024-04-25T11:41:46.386854+0000","last_undegraded":"2024-04-25T11:41:46.386854+0000","last_fullsized":"2024-04-25T11:41:46.386854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.586793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.892455+0000","last_change":"2024-04-25T11:41:44.909936+0000","last_active":"2024-04-25T11:41:45.892455+0000","last_peered":"2024-04-25T11:41:45.892455+0000","last_clean":"2024-04-25T11:41:45.892455+0000","last_became_active":"2024-04-25T11:41:44.909645+0000","last_became_peered":"2024-04-25T11:41:44.909645+0000","last_unstale":"2024-04-25T11:41:45.892455+0000","last_undegraded":"2024-04-25T11:41:45.892455+0000","last_fullsized":"2024-04-25T11:41:45.892455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:12.518017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.602577+0000","last_change":"2024-04-25T11:41:44.913907+0000","last_active":"2024-04-25T11:41:46.602577+0000","last_peered":"2024-04-25T11:41:46.602577+0000","last_clean":"2024-04-25T11:41:46.602577+0000","last_became_active":"2024-04-25T11:41:44.912593+0000","last_became_peered":"2024-04-25T11:41:44.912593+0000","last_unstale":"2024-04-25T11:41:46.602577+0000","last_undegraded":"2024-04-25T11:41:46.602577+0000","last_fullsized":"2024-04-25T11:41:46.602577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:40.164688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388490+0000","last_change":"2024-04-25T11:41:44.903237+0000","last_active":"2024-04-25T11:41:46.388490+0000","last_peered":"2024-04-25T11:41:46.388490+0000","last_clean":"2024-04-25T11:41:46.388490+0000","last_became_active":"2024-04-25T11:41:44.902881+0000","last_became_peered":"2024-04-25T11:41:44.902881+0000","last_unstale":"2024-04-25T11:41:46.388490+0000","last_undegraded":"2024-04-25T11:41:46.388490+0000","last_fullsized":"2024-04-25T11:41:46.388490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:36.201322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388417+0000","last_change":"2024-04-25T11:41:44.907497+0000","last_active":"2024-04-25T11:41:46.388417+0000","last_peered":"2024-04-25T11:41:46.388417+0000","last_clean":"2024-04-25T11:41:46.388417+0000","last_became_active":"2024-04-25T11:41:44.907117+0000","last_became_peered":"2024-04-25T11:41:44.907117+0000","last_unstale":"2024-04-25T11:41:46.388417+0000","last_undegraded":"2024-04-25T11:41:46.388417+0000","last_fullsized":"2024-04-25T11:41:46.388417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:25.987238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.893905+0000","last_change":"2024-04-25T11:41:44.915499+0000","last_active":"2024-04-25T11:41:45.893905+0000","last_peered":"2024-04-25T11:41:45.893905+0000","last_clean":"2024-04-25T11:41:45.893905+0000","last_became_active":"2024-04-25T11:41:44.915221+0000","last_became_peered":"2024-04-25T11:41:44.915221+0000","last_unstale":"2024-04-25T11:41:45.893905+0000","last_undegraded":"2024-04-25T11:41:45.893905+0000","last_fullsized":"2024-04-25T11:41:45.893905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:36.717428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388735+0000","last_change":"2024-04-25T11:41:44.910599+0000","last_active":"2024-04-25T11:41:46.388735+0000","last_peered":"2024-04-25T11:41:46.388735+0000","last_clean":"2024-04-25T11:41:46.388735+0000","last_became_active":"2024-04-25T11:41:44.908747+0000","last_became_peered":"2024-04-25T11:41:44.908747+0000","last_unstale":"2024-04-25T11:41:46.388735+0000","last_undegraded":"2024-04-25T11:41:46.388735+0000","last_fullsized":"2024-04-25T11:41:46.388735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:50.281168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388662+0000","last_change":"2024-04-25T11:41:44.910692+0000","last_active":"2024-04-25T11:41:46.388662+0000","last_peered":"2024-04-25T11:41:46.388662+0000","last_clean":"2024-04-25T11:41:46.388662+0000","last_became_active":"2024-04-25T11:41:44.910343+0000","last_became_peered":"2024-04-25T11:41:44.910343+0000","last_unstale":"2024-04-25T11:41:46.388662+0000","last_undegraded":"2024-04-25T11:41:46.388662+0000","last_fullsized":"2024-04-25T11:41:46.388662+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.945563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387859+0000","last_change":"2024-04-25T11:41:44.910751+0000","last_active":"2024-04-25T11:41:46.387859+0000","last_peered":"2024-04-25T11:41:46.387859+0000","last_clean":"2024-04-25T11:41:46.387859+0000","last_became_active":"2024-04-25T11:41:44.909686+0000","last_became_peered":"2024-04-25T11:41:44.909686+0000","last_unstale":"2024-04-25T11:41:46.387859+0000","last_undegraded":"2024-04-25T11:41:46.387859+0000","last_fullsized":"2024-04-25T11:41:46.387859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:37.216282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.892373+0000","last_change":"2024-04-25T11:41:44.915148+0000","last_active":"2024-04-25T11:41:45.892373+0000","last_peered":"2024-04-25T11:41:45.892373+0000","last_clean":"2024-04-25T11:41:45.892373+0000","last_became_active":"2024-04-25T11:41:44.914772+0000","last_became_peered":"2024-04-25T11:41:44.914772+0000","last_unstale":"2024-04-25T11:41:45.892373+0000","last_undegraded":"2024-04-25T11:41:45.892373+0000","last_fullsized":"2024-04-25T11:41:45.892373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:09.121386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387616+0000","last_change":"2024-04-25T11:41:44.906569+0000","last_active":"2024-04-25T11:41:46.387616+0000","last_peered":"2024-04-25T11:41:46.387616+0000","last_clean":"2024-04-25T11:41:46.387616+0000","last_became_active":"2024-04-25T11:41:44.906259+0000","last_became_peered":"2024-04-25T11:41:44.906259+0000","last_unstale":"2024-04-25T11:41:46.387616+0000","last_undegraded":"2024-04-25T11:41:46.387616+0000","last_fullsized":"2024-04-25T11:41:46.387616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:47.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.386954+0000","last_change":"2024-04-25T11:41:44.909261+0000","last_active":"2024-04-25T11:41:46.386954+0000","last_peered":"2024-04-25T11:41:46.386954+0000","last_clean":"2024-04-25T11:41:46.386954+0000","last_became_active":"2024-04-25T11:41:44.907931+0000","last_became_peered":"2024-04-25T11:41:44.907931+0000","last_unstale":"2024-04-25T11:41:46.386954+0000","last_undegraded":"2024-04-25T11:41:46.386954+0000","last_fullsized":"2024-04-25T11:41:46.386954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:35.754990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.386604+0000","last_change":"2024-04-25T11:41:44.908695+0000","last_active":"2024-04-25T11:41:46.386604+0000","last_peered":"2024-04-25T11:41:46.386604+0000","last_clean":"2024-04-25T11:41:46.386604+0000","last_became_active":"2024-04-25T11:41:44.908374+0000","last_became_peered":"2024-04-25T11:41:44.908374+0000","last_unstale":"2024-04-25T11:41:46.386604+0000","last_undegraded":"2024-04-25T11:41:46.386604+0000","last_fullsized":"2024-04-25T11:41:46.386604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:30.120631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387003+0000","last_change":"2024-04-25T11:41:44.903279+0000","last_active":"2024-04-25T11:41:46.387003+0000","last_peered":"2024-04-25T11:41:46.387003+0000","last_clean":"2024-04-25T11:41:46.387003+0000","last_became_active":"2024-04-25T11:41:44.902958+0000","last_became_peered":"2024-04-25T11:41:44.902958+0000","last_unstale":"2024-04-25T11:41:46.387003+0000","last_undegraded":"2024-04-25T11:41:46.387003+0000","last_fullsized":"2024-04-25T11:41:46.387003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:18.380774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388263+0000","last_change":"2024-04-25T11:41:44.908108+0000","last_active":"2024-04-25T11:41:46.388263+0000","last_peered":"2024-04-25T11:41:46.388263+0000","last_clean":"2024-04-25T11:41:46.388263+0000","last_became_active":"2024-04-25T11:41:44.907787+0000","last_became_peered":"2024-04-25T11:41:44.907787+0000","last_unstale":"2024-04-25T11:41:46.388263+0000","last_undegraded":"2024-04-25T11:41:46.388263+0000","last_fullsized":"2024-04-25T11:41:46.388263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:53.306845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387122+0000","last_change":"2024-04-25T11:41:44.906630+0000","last_active":"2024-04-25T11:41:46.387122+0000","last_peered":"2024-04-25T11:41:46.387122+0000","last_clean":"2024-04-25T11:41:46.387122+0000","last_became_active":"2024-04-25T11:41:44.906321+0000","last_became_peered":"2024-04-25T11:41:44.906321+0000","last_unstale":"2024-04-25T11:41:46.387122+0000","last_undegraded":"2024-04-25T11:41:46.387122+0000","last_fullsized":"2024-04-25T11:41:46.387122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:25.588918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387863+0000","last_change":"2024-04-25T11:41:44.905390+0000","last_active":"2024-04-25T11:41:46.387863+0000","last_peered":"2024-04-25T11:41:46.387863+0000","last_clean":"2024-04-25T11:41:46.387863+0000","last_became_active":"2024-04-25T11:41:44.905033+0000","last_became_peered":"2024-04-25T11:41:44.905033+0000","last_unstale":"2024-04-25T11:41:46.387863+0000","last_undegraded":"2024-04-25T11:41:46.387863+0000","last_fullsized":"2024-04-25T11:41:46.387863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:00.998434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'11","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:49.618681+0000","last_change":"2024-04-25T11:41:44.454718+0000","last_active":"2024-04-25T11:41:49.618681+0000","last_peered":"2024-04-25T11:41:49.618681+0000","last_clean":"2024-04-25T11:41:49.618681+0000","last_became_active":"2024-04-25T11:41:44.454400+0000","last_became_peered":"2024-04-25T11:41:44.454400+0000","last_unstale":"2024-04-25T11:41:49.618681+0000","last_undegraded":"2024-04-25T11:41:49.618681+0000","last_fullsized":"2024-04-25T11:41:49.618681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:05.965988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894254+0000","last_change":"2024-04-25T11:41:44.913512+0000","last_active":"2024-04-25T11:41:45.894254+0000","last_peered":"2024-04-25T11:41:45.894254+0000","last_clean":"2024-04-25T11:41:45.894254+0000","last_became_active":"2024-04-25T11:41:44.913168+0000","last_became_peered":"2024-04-25T11:41:44.913168+0000","last_unstale":"2024-04-25T11:41:45.894254+0000","last_undegraded":"2024-04-25T11:41:45.894254+0000","last_fullsized":"2024-04-25T11:41:45.894254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:25.139747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.429119+0000","last_change":"2024-04-25T11:41:44.451331+0000","last_active":"2024-04-25T11:41:46.429119+0000","last_peered":"2024-04-25T11:41:46.429119+0000","last_clean":"2024-04-25T11:41:46.429119+0000","last_became_active":"2024-04-25T11:41:44.450788+0000","last_became_peered":"2024-04-25T11:41:44.450788+0000","last_unstale":"2024-04-25T11:41:46.429119+0000","last_undegraded":"2024-04-25T11:41:46.429119+0000","last_fullsized":"2024-04-25T11:41:46.429119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:49.011558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387341+0000","last_change":"2024-04-25T11:41:44.908536+0000","last_active":"2024-04-25T11:41:46.387341+0000","last_peered":"2024-04-25T11:41:46.387341+0000","last_clean":"2024-04-25T11:41:46.387341+0000","last_became_active":"2024-04-25T11:41:44.907823+0000","last_became_peered":"2024-04-25T11:41:44.907823+0000","last_unstale":"2024-04-25T11:41:46.387341+0000","last_undegraded":"2024-04-25T11:41:46.387341+0000","last_fullsized":"2024-04-25T11:41:46.387341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:19.898695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.602853+0000","last_change":"2024-04-25T11:41:44.454510+0000","last_active":"2024-04-25T11:41:46.602853+0000","last_peered":"2024-04-25T11:41:46.602853+0000","last_clean":"2024-04-25T11:41:46.602853+0000","last_became_active":"2024-04-25T11:41:44.453452+0000","last_became_peered":"2024-04-25T11:41:44.453452+0000","last_unstale":"2024-04-25T11:41:46.602853+0000","last_undegraded":"2024-04-25T11:41:46.602853+0000","last_fullsized":"2024-04-25T11:41:46.602853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:13.945465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388228+0000","last_change":"2024-04-25T11:41:44.908881+0000","last_active":"2024-04-25T11:41:46.388228+0000","last_peered":"2024-04-25T11:41:46.388228+0000","last_clean":"2024-04-25T11:41:46.388228+0000","last_became_active":"2024-04-25T11:41:44.908493+0000","last_became_peered":"2024-04-25T11:41:44.908493+0000","last_unstale":"2024-04-25T11:41:46.388228+0000","last_undegraded":"2024-04-25T11:41:46.388228+0000","last_fullsized":"2024-04-25T11:41:46.388228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:58.930420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.602575+0000","last_change":"2024-04-25T11:41:44.451870+0000","last_active":"2024-04-25T11:41:46.602575+0000","last_peered":"2024-04-25T11:41:46.602575+0000","last_clean":"2024-04-25T11:41:46.602575+0000","last_became_active":"2024-04-25T11:41:44.451511+0000","last_became_peered":"2024-04-25T11:41:44.451511+0000","last_unstale":"2024-04-25T11:41:46.602575+0000","last_undegraded":"2024-04-25T11:41:46.602575+0000","last_fullsized":"2024-04-25T11:41:46.602575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:26.457919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.895037+0000","last_change":"2024-04-25T11:41:44.910738+0000","last_active":"2024-04-25T11:41:45.895037+0000","last_peered":"2024-04-25T11:41:45.895037+0000","last_clean":"2024-04-25T11:41:45.895037+0000","last_became_active":"2024-04-25T11:41:44.910415+0000","last_became_peered":"2024-04-25T11:41:44.910415+0000","last_unstale":"2024-04-25T11:41:45.895037+0000","last_undegraded":"2024-04-25T11:41:45.895037+0000","last_fullsized":"2024-04-25T11:41:45.895037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:01.333894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.600889+0000","last_change":"2024-04-25T11:41:44.455288+0000","last_active":"2024-04-25T11:41:47.600889+0000","last_peered":"2024-04-25T11:41:47.600889+0000","last_clean":"2024-04-25T11:41:47.600889+0000","last_became_active":"2024-04-25T11:41:44.454965+0000","last_became_peered":"2024-04-25T11:41:44.454965+0000","last_unstale":"2024-04-25T11:41:47.600889+0000","last_undegraded":"2024-04-25T11:41:47.600889+0000","last_fullsized":"2024-04-25T11:41:47.600889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:33.943287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.602512+0000","last_change":"2024-04-25T11:41:44.914274+0000","last_active":"2024-04-25T11:41:46.602512+0000","last_peered":"2024-04-25T11:41:46.602512+0000","last_clean":"2024-04-25T11:41:46.602512+0000","last_became_active":"2024-04-25T11:41:44.913760+0000","last_became_peered":"2024-04-25T11:41:44.913760+0000","last_unstale":"2024-04-25T11:41:46.602512+0000","last_undegraded":"2024-04-25T11:41:46.602512+0000","last_fullsized":"2024-04-25T11:41:46.602512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:57.208895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:48.613484+0000","last_change":"2024-04-25T11:41:44.451846+0000","last_active":"2024-04-25T11:41:48.613484+0000","last_peered":"2024-04-25T11:41:48.613484+0000","last_clean":"2024-04-25T11:41:48.613484+0000","last_became_active":"2024-04-25T11:41:44.451416+0000","last_became_peered":"2024-04-25T11:41:44.451416+0000","last_unstale":"2024-04-25T11:41:48.613484+0000","last_undegraded":"2024-04-25T11:41:48.613484+0000","last_fullsized":"2024-04-25T11:41:48.613484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:57.023883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894981+0000","last_change":"2024-04-25T11:41:44.907761+0000","last_active":"2024-04-25T11:41:45.894981+0000","last_peered":"2024-04-25T11:41:45.894981+0000","last_clean":"2024-04-25T11:41:45.894981+0000","last_became_active":"2024-04-25T11:41:44.906743+0000","last_became_peered":"2024-04-25T11:41:44.906743+0000","last_unstale":"2024-04-25T11:41:45.894981+0000","last_undegraded":"2024-04-25T11:41:45.894981+0000","last_fullsized":"2024-04-25T11:41:45.894981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:43.304985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:48.612944+0000","last_change":"2024-04-25T11:41:44.447415+0000","last_active":"2024-04-25T11:41:48.612944+0000","last_peered":"2024-04-25T11:41:48.612944+0000","last_clean":"2024-04-25T11:41:48.612944+0000","last_became_active":"2024-04-25T11:41:44.447091+0000","last_became_peered":"2024-04-25T11:41:44.447091+0000","last_unstale":"2024-04-25T11:41:48.612944+0000","last_undegraded":"2024-04-25T11:41:48.612944+0000","last_fullsized":"2024-04-25T11:41:48.612944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:27.879115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.893989+0000","last_change":"2024-04-25T11:41:44.913350+0000","last_active":"2024-04-25T11:41:45.893989+0000","last_peered":"2024-04-25T11:41:45.893989+0000","last_clean":"2024-04-25T11:41:45.893989+0000","last_became_active":"2024-04-25T11:41:44.912602+0000","last_became_peered":"2024-04-25T11:41:44.912602+0000","last_unstale":"2024-04-25T11:41:45.893989+0000","last_undegraded":"2024-04-25T11:41:45.893989+0000","last_fullsized":"2024-04-25T11:41:45.893989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:03.099520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894387+0000","last_change":"2024-04-25T11:41:44.451283+0000","last_active":"2024-04-25T11:41:45.894387+0000","last_peered":"2024-04-25T11:41:45.894387+0000","last_clean":"2024-04-25T11:41:45.894387+0000","last_became_active":"2024-04-25T11:41:44.450955+0000","last_became_peered":"2024-04-25T11:41:44.450955+0000","last_unstale":"2024-04-25T11:41:45.894387+0000","last_undegraded":"2024-04-25T11:41:45.894387+0000","last_fullsized":"2024-04-25T11:41:45.894387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:00.912227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387363+0000","last_change":"2024-04-25T11:41:44.904055+0000","last_active":"2024-04-25T11:41:46.387363+0000","last_peered":"2024-04-25T11:41:46.387363+0000","last_clean":"2024-04-25T11:41:46.387363+0000","last_became_active":"2024-04-25T11:41:44.903731+0000","last_became_peered":"2024-04-25T11:41:44.903731+0000","last_unstale":"2024-04-25T11:41:46.387363+0000","last_undegraded":"2024-04-25T11:41:46.387363+0000","last_fullsized":"2024-04-25T11:41:46.387363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:09.912305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.599433+0000","last_change":"2024-04-25T11:41:44.444989+0000","last_active":"2024-04-25T11:41:47.599433+0000","last_peered":"2024-04-25T11:41:47.599433+0000","last_clean":"2024-04-25T11:41:47.599433+0000","last_became_active":"2024-04-25T11:41:44.444543+0000","last_became_peered":"2024-04-25T11:41:44.444543+0000","last_unstale":"2024-04-25T11:41:47.599433+0000","last_undegraded":"2024-04-25T11:41:47.599433+0000","last_fullsized":"2024-04-25T11:41:47.599433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:08.957068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387428+0000","last_change":"2024-04-25T11:41:44.907435+0000","last_active":"2024-04-25T11:41:46.387428+0000","last_peered":"2024-04-25T11:41:46.387428+0000","last_clean":"2024-04-25T11:41:46.387428+0000","last_became_active":"2024-04-25T11:41:44.906972+0000","last_became_peered":"2024-04-25T11:41:44.906972+0000","last_unstale":"2024-04-25T11:41:46.387428+0000","last_undegraded":"2024-04-25T11:41:46.387428+0000","last_fullsized":"2024-04-25T11:41:46.387428+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:47.011440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.600108+0000","last_change":"2024-04-25T11:41:44.456804+0000","last_active":"2024-04-25T11:41:47.600108+0000","last_peered":"2024-04-25T11:41:47.600108+0000","last_clean":"2024-04-25T11:41:47.600108+0000","last_became_active":"2024-04-25T11:41:44.456472+0000","last_became_peered":"2024-04-25T11:41:44.456472+0000","last_unstale":"2024-04-25T11:41:47.600108+0000","last_undegraded":"2024-04-25T11:41:47.600108+0000","last_fullsized":"2024-04-25T11:41:47.600108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:49.293772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387317+0000","last_change":"2024-04-25T11:41:44.912003+0000","last_active":"2024-04-25T11:41:46.387317+0000","last_peered":"2024-04-25T11:41:46.387317+0000","last_clean":"2024-04-25T11:41:46.387317+0000","last_became_active":"2024-04-25T11:41:44.909923+0000","last_became_peered":"2024-04-25T11:41:44.909923+0000","last_unstale":"2024-04-25T11:41:46.387317+0000","last_undegraded":"2024-04-25T11:41:46.387317+0000","last_fullsized":"2024-04-25T11:41:46.387317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:56.631924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894971+0000","last_change":"2024-04-25T11:41:44.453860+0000","last_active":"2024-04-25T11:41:45.894971+0000","last_peered":"2024-04-25T11:41:45.894971+0000","last_clean":"2024-04-25T11:41:45.894971+0000","last_became_active":"2024-04-25T11:41:44.453536+0000","last_became_peered":"2024-04-25T11:41:44.453536+0000","last_unstale":"2024-04-25T11:41:45.894971+0000","last_undegraded":"2024-04-25T11:41:45.894971+0000","last_fullsized":"2024-04-25T11:41:45.894971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:56.762115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388444+0000","last_change":"2024-04-25T11:41:44.913443+0000","last_active":"2024-04-25T11:41:46.388444+0000","last_peered":"2024-04-25T11:41:46.388444+0000","last_clean":"2024-04-25T11:41:46.388444+0000","last_became_active":"2024-04-25T11:41:44.911350+0000","last_became_peered":"2024-04-25T11:41:44.911350+0000","last_unstale":"2024-04-25T11:41:46.388444+0000","last_undegraded":"2024-04-25T11:41:46.388444+0000","last_fullsized":"2024-04-25T11:41:46.388444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:58.345587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.598911+0000","last_change":"2024-04-25T11:41:44.452286+0000","last_active":"2024-04-25T11:41:47.598911+0000","last_peered":"2024-04-25T11:41:47.598911+0000","last_clean":"2024-04-25T11:41:47.598911+0000","last_became_active":"2024-04-25T11:41:44.451928+0000","last_became_peered":"2024-04-25T11:41:44.451928+0000","last_unstale":"2024-04-25T11:41:47.598911+0000","last_undegraded":"2024-04-25T11:41:47.598911+0000","last_fullsized":"2024-04-25T11:41:47.598911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:21.407375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.602726+0000","last_change":"2024-04-25T11:41:44.903909+0000","last_active":"2024-04-25T11:41:46.602726+0000","last_peered":"2024-04-25T11:41:46.602726+0000","last_clean":"2024-04-25T11:41:46.602726+0000","last_became_active":"2024-04-25T11:41:44.903620+0000","last_became_peered":"2024-04-25T11:41:44.903620+0000","last_unstale":"2024-04-25T11:41:46.602726+0000","last_undegraded":"2024-04-25T11:41:46.602726+0000","last_fullsized":"2024-04-25T11:41:46.602726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:49.910052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:51.584739+0000","last_change":"2024-04-25T11:41:44.454294+0000","last_active":"2024-04-25T11:41:51.584739+0000","last_peered":"2024-04-25T11:41:51.584739+0000","last_clean":"2024-04-25T11:41:51.584739+0000","last_became_active":"2024-04-25T11:41:44.453829+0000","last_became_peered":"2024-04-25T11:41:44.453829+0000","last_unstale":"2024-04-25T11:41:51.584739+0000","last_undegraded":"2024-04-25T11:41:51.584739+0000","last_fullsized":"2024-04-25T11:41:51.584739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:14.617963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.386726+0000","last_change":"2024-04-25T11:41:44.907453+0000","last_active":"2024-04-25T11:41:46.386726+0000","last_peered":"2024-04-25T11:41:46.386726+0000","last_clean":"2024-04-25T11:41:46.386726+0000","last_became_active":"2024-04-25T11:41:44.907126+0000","last_became_peered":"2024-04-25T11:41:44.907126+0000","last_unstale":"2024-04-25T11:41:46.386726+0000","last_undegraded":"2024-04-25T11:41:46.386726+0000","last_fullsized":"2024-04-25T11:41:46.386726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:25.998686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.599807+0000","last_change":"2024-04-25T11:41:44.452258+0000","last_active":"2024-04-25T11:41:47.599807+0000","last_peered":"2024-04-25T11:41:47.599807+0000","last_clean":"2024-04-25T11:41:47.599807+0000","last_became_active":"2024-04-25T11:41:44.451880+0000","last_became_peered":"2024-04-25T11:41:44.451880+0000","last_unstale":"2024-04-25T11:41:47.599807+0000","last_undegraded":"2024-04-25T11:41:47.599807+0000","last_fullsized":"2024-04-25T11:41:47.599807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:39.463406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387631+0000","last_change":"2024-04-25T11:41:44.913841+0000","last_active":"2024-04-25T11:41:46.387631+0000","last_peered":"2024-04-25T11:41:46.387631+0000","last_clean":"2024-04-25T11:41:46.387631+0000","last_became_active":"2024-04-25T11:41:44.913378+0000","last_became_peered":"2024-04-25T11:41:44.913378+0000","last_unstale":"2024-04-25T11:41:46.387631+0000","last_undegraded":"2024-04-25T11:41:46.387631+0000","last_fullsized":"2024-04-25T11:41:46.387631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:24.732899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:49.617003+0000","last_change":"2024-04-25T11:41:44.450531+0000","last_active":"2024-04-25T11:41:49.617003+0000","last_peered":"2024-04-25T11:41:49.617003+0000","last_clean":"2024-04-25T11:41:49.617003+0000","last_became_active":"2024-04-25T11:41:44.450205+0000","last_became_peered":"2024-04-25T11:41:44.450205+0000","last_unstale":"2024-04-25T11:41:49.617003+0000","last_undegraded":"2024-04-25T11:41:49.617003+0000","last_fullsized":"2024-04-25T11:41:49.617003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:51.797969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.386972+0000","last_change":"2024-04-25T11:41:44.911883+0000","last_active":"2024-04-25T11:41:46.386972+0000","last_peered":"2024-04-25T11:41:46.386972+0000","last_clean":"2024-04-25T11:41:46.386972+0000","last_became_active":"2024-04-25T11:41:44.911362+0000","last_became_peered":"2024-04-25T11:41:44.911362+0000","last_unstale":"2024-04-25T11:41:46.386972+0000","last_undegraded":"2024-04-25T11:41:46.386972+0000","last_fullsized":"2024-04-25T11:41:46.386972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:34.273876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'10","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.599347+0000","last_change":"2024-04-25T11:41:44.455844+0000","last_active":"2024-04-25T11:41:47.599347+0000","last_peered":"2024-04-25T11:41:47.599347+0000","last_clean":"2024-04-25T11:41:47.599347+0000","last_became_active":"2024-04-25T11:41:44.455167+0000","last_became_peered":"2024-04-25T11:41:44.455167+0000","last_unstale":"2024-04-25T11:41:47.599347+0000","last_undegraded":"2024-04-25T11:41:47.599347+0000","last_fullsized":"2024-04-25T11:41:47.599347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.886078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894810+0000","last_change":"2024-04-25T11:41:44.912145+0000","last_active":"2024-04-25T11:41:45.894810+0000","last_peered":"2024-04-25T11:41:45.894810+0000","last_clean":"2024-04-25T11:41:45.894810+0000","last_became_active":"2024-04-25T11:41:44.911113+0000","last_became_peered":"2024-04-25T11:41:44.911113+0000","last_unstale":"2024-04-25T11:41:45.894810+0000","last_undegraded":"2024-04-25T11:41:45.894810+0000","last_fullsized":"2024-04-25T11:41:45.894810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:35.983358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.602818+0000","last_change":"2024-04-25T11:41:44.452371+0000","last_active":"2024-04-25T11:41:46.602818+0000","last_peered":"2024-04-25T11:41:46.602818+0000","last_clean":"2024-04-25T11:41:46.602818+0000","last_became_active":"2024-04-25T11:41:44.451406+0000","last_became_peered":"2024-04-25T11:41:44.451406+0000","last_unstale":"2024-04-25T11:41:46.602818+0000","last_undegraded":"2024-04-25T11:41:46.602818+0000","last_fullsized":"2024-04-25T11:41:46.602818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:00.093835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387947+0000","last_change":"2024-04-25T11:41:44.910999+0000","last_active":"2024-04-25T11:41:46.387947+0000","last_peered":"2024-04-25T11:41:46.387947+0000","last_clean":"2024-04-25T11:41:46.387947+0000","last_became_active":"2024-04-25T11:41:44.909749+0000","last_became_peered":"2024-04-25T11:41:44.909749+0000","last_unstale":"2024-04-25T11:41:46.387947+0000","last_undegraded":"2024-04-25T11:41:46.387947+0000","last_fullsized":"2024-04-25T11:41:46.387947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:44.272145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387031+0000","last_change":"2024-04-25T11:41:44.453243+0000","last_active":"2024-04-25T11:41:46.387031+0000","last_peered":"2024-04-25T11:41:46.387031+0000","last_clean":"2024-04-25T11:41:46.387031+0000","last_became_active":"2024-04-25T11:41:44.452925+0000","last_became_peered":"2024-04-25T11:41:44.452925+0000","last_unstale":"2024-04-25T11:41:46.387031+0000","last_undegraded":"2024-04-25T11:41:46.387031+0000","last_fullsized":"2024-04-25T11:41:46.387031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:57.918332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388274+0000","last_change":"2024-04-25T11:41:44.907350+0000","last_active":"2024-04-25T11:41:46.388274+0000","last_peered":"2024-04-25T11:41:46.388274+0000","last_clean":"2024-04-25T11:41:46.388274+0000","last_became_active":"2024-04-25T11:41:44.907000+0000","last_became_peered":"2024-04-25T11:41:44.907000+0000","last_unstale":"2024-04-25T11:41:46.388274+0000","last_undegraded":"2024-04-25T11:41:46.388274+0000","last_fullsized":"2024-04-25T11:41:46.388274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:08.823550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387290+0000","last_change":"2024-04-25T11:41:44.455318+0000","last_active":"2024-04-25T11:41:46.387290+0000","last_peered":"2024-04-25T11:41:46.387290+0000","last_clean":"2024-04-25T11:41:46.387290+0000","last_became_active":"2024-04-25T11:41:44.454972+0000","last_became_peered":"2024-04-25T11:41:44.454972+0000","last_unstale":"2024-04-25T11:41:46.387290+0000","last_undegraded":"2024-04-25T11:41:46.387290+0000","last_fullsized":"2024-04-25T11:41:46.387290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.255860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388648+0000","last_change":"2024-04-25T11:41:44.912507+0000","last_active":"2024-04-25T11:41:46.388648+0000","last_peered":"2024-04-25T11:41:46.388648+0000","last_clean":"2024-04-25T11:41:46.388648+0000","last_became_active":"2024-04-25T11:41:44.912123+0000","last_became_peered":"2024-04-25T11:41:44.912123+0000","last_unstale":"2024-04-25T11:41:46.388648+0000","last_undegraded":"2024-04-25T11:41:46.388648+0000","last_fullsized":"2024-04-25T11:41:46.388648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:40.659961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894716+0000","last_change":"2024-04-25T11:41:44.913505+0000","last_active":"2024-04-25T11:41:45.894716+0000","last_peered":"2024-04-25T11:41:45.894716+0000","last_clean":"2024-04-25T11:41:45.894716+0000","last_became_active":"2024-04-25T11:41:44.913103+0000","last_became_peered":"2024-04-25T11:41:44.913103+0000","last_unstale":"2024-04-25T11:41:45.894716+0000","last_undegraded":"2024-04-25T11:41:45.894716+0000","last_fullsized":"2024-04-25T11:41:45.894716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:49.414802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:48.613838+0000","last_change":"2024-04-25T11:41:44.450056+0000","last_active":"2024-04-25T11:41:48.613838+0000","last_peered":"2024-04-25T11:41:48.613838+0000","last_clean":"2024-04-25T11:41:48.613838+0000","last_became_active":"2024-04-25T11:41:44.449754+0000","last_became_peered":"2024-04-25T11:41:44.449754+0000","last_unstale":"2024-04-25T11:41:48.613838+0000","last_undegraded":"2024-04-25T11:41:48.613838+0000","last_fullsized":"2024-04-25T11:41:48.613838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:30.059852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894315+0000","last_change":"2024-04-25T11:41:44.915061+0000","last_active":"2024-04-25T11:41:45.894315+0000","last_peered":"2024-04-25T11:41:45.894315+0000","last_clean":"2024-04-25T11:41:45.894315+0000","last_became_active":"2024-04-25T11:41:44.914799+0000","last_became_peered":"2024-04-25T11:41:44.914799+0000","last_unstale":"2024-04-25T11:41:45.894315+0000","last_undegraded":"2024-04-25T11:41:45.894315+0000","last_fullsized":"2024-04-25T11:41:45.894315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:22.820539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:49.616317+0000","last_change":"2024-04-25T11:41:44.449248+0000","last_active":"2024-04-25T11:41:49.616317+0000","last_peered":"2024-04-25T11:41:49.616317+0000","last_clean":"2024-04-25T11:41:49.616317+0000","last_became_active":"2024-04-25T11:41:44.448898+0000","last_became_peered":"2024-04-25T11:41:44.448898+0000","last_unstale":"2024-04-25T11:41:49.616317+0000","last_undegraded":"2024-04-25T11:41:49.616317+0000","last_fullsized":"2024-04-25T11:41:49.616317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:19.574270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387095+0000","last_change":"2024-04-25T11:40:36.892561+0000","last_active":"2024-04-25T11:41:46.387095+0000","last_peered":"2024-04-25T11:41:46.387095+0000","last_clean":"2024-04-25T11:41:46.387095+0000","last_became_active":"2024-04-25T11:40:36.892126+0000","last_became_peered":"2024-04-25T11:40:36.892126+0000","last_unstale":"2024-04-25T11:41:46.387095+0000","last_undegraded":"2024-04-25T11:41:46.387095+0000","last_fullsized":"2024-04-25T11:41:46.387095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:37:16.666579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:37:16.666579+0000","last_clean_scrub_stamp":"2024-04-25T11:37:16.666579+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:44.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.386972+0000","last_change":"2024-04-25T11:41:44.908446+0000","last_active":"2024-04-25T11:41:46.386972+0000","last_peered":"2024-04-25T11:41:46.386972+0000","last_clean":"2024-04-25T11:41:46.386972+0000","last_became_active":"2024-04-25T11:41:44.908148+0000","last_became_peered":"2024-04-25T11:41:44.908148+0000","last_unstale":"2024-04-25T11:41:46.386972+0000","last_undegraded":"2024-04-25T11:41:46.386972+0000","last_fullsized":"2024-04-25T11:41:46.386972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:09.598976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:48.612514+0000","last_change":"2024-04-25T11:41:44.454815+0000","last_active":"2024-04-25T11:41:48.612514+0000","last_peered":"2024-04-25T11:41:48.612514+0000","last_clean":"2024-04-25T11:41:48.612514+0000","last_became_active":"2024-04-25T11:41:44.454521+0000","last_became_peered":"2024-04-25T11:41:44.454521+0000","last_unstale":"2024-04-25T11:41:48.612514+0000","last_undegraded":"2024-04-25T11:41:48.612514+0000","last_fullsized":"2024-04-25T11:41:48.612514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:45.932329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.599233+0000","last_change":"2024-04-25T11:41:44.456641+0000","last_active":"2024-04-25T11:41:47.599233+0000","last_peered":"2024-04-25T11:41:47.599233+0000","last_clean":"2024-04-25T11:41:47.599233+0000","last_became_active":"2024-04-25T11:41:44.456133+0000","last_became_peered":"2024-04-25T11:41:44.456133+0000","last_unstale":"2024-04-25T11:41:47.599233+0000","last_undegraded":"2024-04-25T11:41:47.599233+0000","last_fullsized":"2024-04-25T11:41:47.599233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:47.746160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387484+0000","last_change":"2024-04-25T11:41:44.897512+0000","last_active":"2024-04-25T11:41:46.387484+0000","last_peered":"2024-04-25T11:41:46.387484+0000","last_clean":"2024-04-25T11:41:46.387484+0000","last_became_active":"2024-04-25T11:41:44.897184+0000","last_became_peered":"2024-04-25T11:41:44.897184+0000","last_unstale":"2024-04-25T11:41:46.387484+0000","last_undegraded":"2024-04-25T11:41:46.387484+0000","last_fullsized":"2024-04-25T11:41:46.387484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:53.703825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:49.619010+0000","last_change":"2024-04-25T11:41:44.458318+0000","last_active":"2024-04-25T11:41:49.619010+0000","last_peered":"2024-04-25T11:41:49.619010+0000","last_clean":"2024-04-25T11:41:49.619010+0000","last_became_active":"2024-04-25T11:41:44.458001+0000","last_became_peered":"2024-04-25T11:41:44.458001+0000","last_unstale":"2024-04-25T11:41:49.619010+0000","last_undegraded":"2024-04-25T11:41:49.619010+0000","last_fullsized":"2024-04-25T11:41:49.619010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:17.100621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894447+0000","last_change":"2024-04-25T11:41:44.905912+0000","last_active":"2024-04-25T11:41:45.894447+0000","last_peered":"2024-04-25T11:41:45.894447+0000","last_clean":"2024-04-25T11:41:45.894447+0000","last_became_active":"2024-04-25T11:41:44.905606+0000","last_became_peered":"2024-04-25T11:41:44.905606+0000","last_unstale":"2024-04-25T11:41:45.894447+0000","last_undegraded":"2024-04-25T11:41:45.894447+0000","last_fullsized":"2024-04-25T11:41:45.894447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:47.609884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.598152+0000","last_change":"2024-04-25T11:41:44.460492+0000","last_active":"2024-04-25T11:41:47.598152+0000","last_peered":"2024-04-25T11:41:47.598152+0000","last_clean":"2024-04-25T11:41:47.598152+0000","last_became_active":"2024-04-25T11:41:44.460199+0000","last_became_peered":"2024-04-25T11:41:44.460199+0000","last_unstale":"2024-04-25T11:41:47.598152+0000","last_undegraded":"2024-04-25T11:41:47.598152+0000","last_fullsized":"2024-04-25T11:41:47.598152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:18.756779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387236+0000","last_change":"2024-04-25T11:41:44.909438+0000","last_active":"2024-04-25T11:41:46.387236+0000","last_peered":"2024-04-25T11:41:46.387236+0000","last_clean":"2024-04-25T11:41:46.387236+0000","last_became_active":"2024-04-25T11:41:44.909146+0000","last_became_peered":"2024-04-25T11:41:44.909146+0000","last_unstale":"2024-04-25T11:41:46.387236+0000","last_undegraded":"2024-04-25T11:41:46.387236+0000","last_fullsized":"2024-04-25T11:41:46.387236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:15.949652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.598441+0000","last_change":"2024-04-25T11:41:44.453169+0000","last_active":"2024-04-25T11:41:47.598441+0000","last_peered":"2024-04-25T11:41:47.598441+0000","last_clean":"2024-04-25T11:41:47.598441+0000","last_became_active":"2024-04-25T11:41:44.452762+0000","last_became_peered":"2024-04-25T11:41:44.452762+0000","last_unstale":"2024-04-25T11:41:47.598441+0000","last_undegraded":"2024-04-25T11:41:47.598441+0000","last_fullsized":"2024-04-25T11:41:47.598441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:13.084197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.602420+0000","last_change":"2024-04-25T11:41:44.910810+0000","last_active":"2024-04-25T11:41:46.602420+0000","last_peered":"2024-04-25T11:41:46.602420+0000","last_clean":"2024-04-25T11:41:46.602420+0000","last_became_active":"2024-04-25T11:41:44.910488+0000","last_became_peered":"2024-04-25T11:41:44.910488+0000","last_unstale":"2024-04-25T11:41:46.602420+0000","last_undegraded":"2024-04-25T11:41:46.602420+0000","last_fullsized":"2024-04-25T11:41:46.602420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:36.971289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:50.622008+0000","last_change":"2024-04-25T11:41:44.455402+0000","last_active":"2024-04-25T11:41:50.622008+0000","last_peered":"2024-04-25T11:41:50.622008+0000","last_clean":"2024-04-25T11:41:50.622008+0000","last_became_active":"2024-04-25T11:41:44.455057+0000","last_became_peered":"2024-04-25T11:41:44.455057+0000","last_unstale":"2024-04-25T11:41:50.622008+0000","last_undegraded":"2024-04-25T11:41:50.622008+0000","last_fullsized":"2024-04-25T11:41:50.622008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:32.156664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387346+0000","last_change":"2024-04-25T11:41:44.903907+0000","last_active":"2024-04-25T11:41:46.387346+0000","last_peered":"2024-04-25T11:41:46.387346+0000","last_clean":"2024-04-25T11:41:46.387346+0000","last_became_active":"2024-04-25T11:41:44.903510+0000","last_became_peered":"2024-04-25T11:41:44.903510+0000","last_unstale":"2024-04-25T11:41:46.387346+0000","last_undegraded":"2024-04-25T11:41:46.387346+0000","last_fullsized":"2024-04-25T11:41:46.387346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:33.183412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:49.618241+0000","last_change":"2024-04-25T11:41:44.454049+0000","last_active":"2024-04-25T11:41:49.618241+0000","last_peered":"2024-04-25T11:41:49.618241+0000","last_clean":"2024-04-25T11:41:49.618241+0000","last_became_active":"2024-04-25T11:41:44.453652+0000","last_became_peered":"2024-04-25T11:41:44.453652+0000","last_unstale":"2024-04-25T11:41:49.618241+0000","last_undegraded":"2024-04-25T11:41:49.618241+0000","last_fullsized":"2024-04-25T11:41:49.618241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:17.239715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894156+0000","last_change":"2024-04-25T11:41:44.915005+0000","last_active":"2024-04-25T11:41:45.894156+0000","last_peered":"2024-04-25T11:41:45.894156+0000","last_clean":"2024-04-25T11:41:45.894156+0000","last_became_active":"2024-04-25T11:41:44.914663+0000","last_became_peered":"2024-04-25T11:41:44.914663+0000","last_unstale":"2024-04-25T11:41:45.894156+0000","last_undegraded":"2024-04-25T11:41:45.894156+0000","last_fullsized":"2024-04-25T11:41:45.894156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:49.390846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:47.600185+0000","last_change":"2024-04-25T11:41:44.454194+0000","last_active":"2024-04-25T11:41:47.600185+0000","last_peered":"2024-04-25T11:41:47.600185+0000","last_clean":"2024-04-25T11:41:47.600185+0000","last_became_active":"2024-04-25T11:41:44.453280+0000","last_became_peered":"2024-04-25T11:41:44.453280+0000","last_unstale":"2024-04-25T11:41:47.600185+0000","last_undegraded":"2024-04-25T11:41:47.600185+0000","last_fullsized":"2024-04-25T11:41:47.600185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:45.368132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387371+0000","last_change":"2024-04-25T11:41:44.907951+0000","last_active":"2024-04-25T11:41:46.387371+0000","last_peered":"2024-04-25T11:41:46.387371+0000","last_clean":"2024-04-25T11:41:46.387371+0000","last_became_active":"2024-04-25T11:41:44.907554+0000","last_became_peered":"2024-04-25T11:41:44.907554+0000","last_unstale":"2024-04-25T11:41:46.387371+0000","last_undegraded":"2024-04-25T11:41:46.387371+0000","last_fullsized":"2024-04-25T11:41:46.387371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:33.312265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:51.588428+0000","last_change":"2024-04-25T11:41:44.458803+0000","last_active":"2024-04-25T11:41:51.588428+0000","last_peered":"2024-04-25T11:41:51.588428+0000","last_clean":"2024-04-25T11:41:51.588428+0000","last_became_active":"2024-04-25T11:41:44.458307+0000","last_became_peered":"2024-04-25T11:41:44.458307+0000","last_unstale":"2024-04-25T11:41:51.588428+0000","last_undegraded":"2024-04-25T11:41:51.588428+0000","last_fullsized":"2024-04-25T11:41:51.588428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:26.186429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387897+0000","last_change":"2024-04-25T11:41:44.909600+0000","last_active":"2024-04-25T11:41:46.387897+0000","last_peered":"2024-04-25T11:41:46.387897+0000","last_clean":"2024-04-25T11:41:46.387897+0000","last_became_active":"2024-04-25T11:41:44.909083+0000","last_became_peered":"2024-04-25T11:41:44.909083+0000","last_unstale":"2024-04-25T11:41:46.387897+0000","last_undegraded":"2024-04-25T11:41:46.387897+0000","last_fullsized":"2024-04-25T11:41:46.387897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:53.854631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894690+0000","last_change":"2024-04-25T11:41:44.451466+0000","last_active":"2024-04-25T11:41:45.894690+0000","last_peered":"2024-04-25T11:41:45.894690+0000","last_clean":"2024-04-25T11:41:45.894690+0000","last_became_active":"2024-04-25T11:41:44.450866+0000","last_became_peered":"2024-04-25T11:41:44.450866+0000","last_unstale":"2024-04-25T11:41:45.894690+0000","last_undegraded":"2024-04-25T11:41:45.894690+0000","last_fullsized":"2024-04-25T11:41:45.894690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:03.715468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387474+0000","last_change":"2024-04-25T11:41:44.909789+0000","last_active":"2024-04-25T11:41:46.387474+0000","last_peered":"2024-04-25T11:41:46.387474+0000","last_clean":"2024-04-25T11:41:46.387474+0000","last_became_active":"2024-04-25T11:41:44.909490+0000","last_became_peered":"2024-04-25T11:41:44.909490+0000","last_unstale":"2024-04-25T11:41:46.387474+0000","last_undegraded":"2024-04-25T11:41:46.387474+0000","last_fullsized":"2024-04-25T11:41:46.387474+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:50.543847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387146+0000","last_change":"2024-04-25T11:41:44.449371+0000","last_active":"2024-04-25T11:41:46.387146+0000","last_peered":"2024-04-25T11:41:46.387146+0000","last_clean":"2024-04-25T11:41:46.387146+0000","last_became_active":"2024-04-25T11:41:44.448927+0000","last_became_peered":"2024-04-25T11:41:44.448927+0000","last_unstale":"2024-04-25T11:41:46.387146+0000","last_undegraded":"2024-04-25T11:41:46.387146+0000","last_fullsized":"2024-04-25T11:41:46.387146+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:49.581010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388120+0000","last_change":"2024-04-25T11:41:44.906540+0000","last_active":"2024-04-25T11:41:46.388120+0000","last_peered":"2024-04-25T11:41:46.388120+0000","last_clean":"2024-04-25T11:41:46.388120+0000","last_became_active":"2024-04-25T11:41:44.905918+0000","last_became_peered":"2024-04-25T11:41:44.905918+0000","last_unstale":"2024-04-25T11:41:46.388120+0000","last_undegraded":"2024-04-25T11:41:46.388120+0000","last_fullsized":"2024-04-25T11:41:46.388120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:38.350461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387314+0000","last_change":"2024-04-25T11:41:44.904656+0000","last_active":"2024-04-25T11:41:46.387314+0000","last_peered":"2024-04-25T11:41:46.387314+0000","last_clean":"2024-04-25T11:41:46.387314+0000","last_became_active":"2024-04-25T11:41:44.904322+0000","last_became_peered":"2024-04-25T11:41:44.904322+0000","last_unstale":"2024-04-25T11:41:46.387314+0000","last_undegraded":"2024-04-25T11:41:46.387314+0000","last_fullsized":"2024-04-25T11:41:46.387314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:07.762816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388192+0000","last_change":"2024-04-25T11:41:44.911662+0000","last_active":"2024-04-25T11:41:46.388192+0000","last_peered":"2024-04-25T11:41:46.388192+0000","last_clean":"2024-04-25T11:41:46.388192+0000","last_became_active":"2024-04-25T11:41:44.911295+0000","last_became_peered":"2024-04-25T11:41:44.911295+0000","last_unstale":"2024-04-25T11:41:46.388192+0000","last_undegraded":"2024-04-25T11:41:46.388192+0000","last_fullsized":"2024-04-25T11:41:46.388192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:12.988118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387426+0000","last_change":"2024-04-25T11:41:44.906303+0000","last_active":"2024-04-25T11:41:46.387426+0000","last_peered":"2024-04-25T11:41:46.387426+0000","last_clean":"2024-04-25T11:41:46.387426+0000","last_became_active":"2024-04-25T11:41:44.905951+0000","last_became_peered":"2024-04-25T11:41:44.905951+0000","last_unstale":"2024-04-25T11:41:46.387426+0000","last_undegraded":"2024-04-25T11:41:46.387426+0000","last_fullsized":"2024-04-25T11:41:46.387426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:49.887856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387090+0000","last_change":"2024-04-25T11:41:44.910457+0000","last_active":"2024-04-25T11:41:46.387090+0000","last_peered":"2024-04-25T11:41:46.387090+0000","last_clean":"2024-04-25T11:41:46.387090+0000","last_became_active":"2024-04-25T11:41:44.909089+0000","last_became_peered":"2024-04-25T11:41:44.909089+0000","last_unstale":"2024-04-25T11:41:46.387090+0000","last_undegraded":"2024-04-25T11:41:46.387090+0000","last_fullsized":"2024-04-25T11:41:46.387090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:25.222918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387530+0000","last_change":"2024-04-25T11:41:44.909391+0000","last_active":"2024-04-25T11:41:46.387530+0000","last_peered":"2024-04-25T11:41:46.387530+0000","last_clean":"2024-04-25T11:41:46.387530+0000","last_became_active":"2024-04-25T11:41:44.909088+0000","last_became_peered":"2024-04-25T11:41:44.909088+0000","last_unstale":"2024-04-25T11:41:46.387530+0000","last_undegraded":"2024-04-25T11:41:46.387530+0000","last_fullsized":"2024-04-25T11:41:46.387530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:38.550716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388028+0000","last_change":"2024-04-25T11:41:44.904351+0000","last_active":"2024-04-25T11:41:46.388028+0000","last_peered":"2024-04-25T11:41:46.388028+0000","last_clean":"2024-04-25T11:41:46.388028+0000","last_became_active":"2024-04-25T11:41:44.904015+0000","last_became_peered":"2024-04-25T11:41:44.904015+0000","last_unstale":"2024-04-25T11:41:46.388028+0000","last_undegraded":"2024-04-25T11:41:46.388028+0000","last_fullsized":"2024-04-25T11:41:46.388028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:18.382370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.388125+0000","last_change":"2024-04-25T11:41:44.901469+0000","last_active":"2024-04-25T11:41:46.388125+0000","last_peered":"2024-04-25T11:41:46.388125+0000","last_clean":"2024-04-25T11:41:46.388125+0000","last_became_active":"2024-04-25T11:41:44.900943+0000","last_became_peered":"2024-04-25T11:41:44.900943+0000","last_unstale":"2024-04-25T11:41:46.388125+0000","last_undegraded":"2024-04-25T11:41:46.388125+0000","last_fullsized":"2024-04-25T11:41:46.388125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:33.712132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894089+0000","last_change":"2024-04-25T11:41:44.912267+0000","last_active":"2024-04-25T11:41:45.894089+0000","last_peered":"2024-04-25T11:41:45.894089+0000","last_clean":"2024-04-25T11:41:45.894089+0000","last_became_active":"2024-04-25T11:41:44.911968+0000","last_became_peered":"2024-04-25T11:41:44.911968+0000","last_unstale":"2024-04-25T11:41:45.894089+0000","last_undegraded":"2024-04-25T11:41:45.894089+0000","last_fullsized":"2024-04-25T11:41:45.894089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:27.956046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.894613+0000","last_change":"2024-04-25T11:41:44.907540+0000","last_active":"2024-04-25T11:41:45.894613+0000","last_peered":"2024-04-25T11:41:45.894613+0000","last_clean":"2024-04-25T11:41:45.894613+0000","last_became_active":"2024-04-25T11:41:44.907229+0000","last_became_peered":"2024-04-25T11:41:44.907229+0000","last_unstale":"2024-04-25T11:41:45.894613+0000","last_undegraded":"2024-04-25T11:41:45.894613+0000","last_fullsized":"2024-04-25T11:41:45.894613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:49.123196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:45.893930+0000","last_change":"2024-04-25T11:41:44.912354+0000","last_active":"2024-04-25T11:41:45.893930+0000","last_peered":"2024-04-25T11:41:45.893930+0000","last_clean":"2024-04-25T11:41:45.893930+0000","last_became_active":"2024-04-25T11:41:44.912028+0000","last_became_peered":"2024-04-25T11:41:44.912028+0000","last_unstale":"2024-04-25T11:41:45.893930+0000","last_undegraded":"2024-04-25T11:41:45.893930+0000","last_fullsized":"2024-04-25T11:41:45.893930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:50.824647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387845+0000","last_change":"2024-04-25T11:41:44.907346+0000","last_active":"2024-04-25T11:41:46.387845+0000","last_peered":"2024-04-25T11:41:46.387845+0000","last_clean":"2024-04-25T11:41:46.387845+0000","last_became_active":"2024-04-25T11:41:44.907027+0000","last_became_peered":"2024-04-25T11:41:44.907027+0000","last_unstale":"2024-04-25T11:41:46.387845+0000","last_undegraded":"2024-04-25T11:41:46.387845+0000","last_fullsized":"2024-04-25T11:41:46.387845+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:01.913218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387532+0000","last_change":"2024-04-25T11:41:44.902606+0000","last_active":"2024-04-25T11:41:46.387532+0000","last_peered":"2024-04-25T11:41:46.387532+0000","last_clean":"2024-04-25T11:41:46.387532+0000","last_became_active":"2024-04-25T11:41:44.902306+0000","last_became_peered":"2024-04-25T11:41:44.902306+0000","last_unstale":"2024-04-25T11:41:46.387532+0000","last_undegraded":"2024-04-25T11:41:46.387532+0000","last_fullsized":"2024-04-25T11:41:46.387532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:24.214013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T11:41:46.387894+0000","last_change":"2024-04-25T11:41:44.909653+0000","last_active":"2024-04-25T11:41:46.387894+0000","last_peered":"2024-04-25T11:41:46.387894+0000","last_clean":"2024-04-25T11:41:46.387894+0000","last_became_active":"2024-04-25T11:41:44.909256+0000","last_became_peered":"2024-04-25T11:41:44.909256+0000","last_unstale":"2024-04-25T11:41:46.387894+0000","last_undegraded":"2024-04-25T11:41:46.387894+0000","last_fullsized":"2024-04-25T11:41:46.387894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:30.310432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6145,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":146,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":299008,"data_stored":53707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350988,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":442151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36852,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708300,"statfs":{"total":95995035648,"available":95957299200,"internally_reserved":0,"allocated":1552384,"data_stored":1035087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":442969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037787,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":442755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299425,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36832,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708320,"statfs":{"total":95995035648,"available":95957319680,"internally_reserved":0,"allocated":1531904,"data_stored":1034311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462950,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":925696,"data_stored":442241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":443763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":442845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411385,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":465409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":954368,"data_stored":442698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888112,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":442795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724586,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1544192,"data_stored":1033995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":40556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:41:52.711 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:41:52.711 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T11:41:52.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T11:41:52.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:52 smithi083 ceph-mon[29330]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:52.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:52 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3842458307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:52.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:52 smithi083 ceph-mon[29330]: pgmap v287: 97 pgs: 97 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 548 B/s rd, 4.6 KiB/s wr, 20 op/s 2024-04-25T11:41:52.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:52 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3373894701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:52.985 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:41:53.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:52 smithi094 ceph-mon[33032]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:53.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:52 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3842458307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:53.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:52 smithi094 ceph-mon[33032]: pgmap v287: 97 pgs: 97 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 548 B/s rd, 4.6 KiB/s wr, 20 op/s 2024-04-25T11:41:53.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:52 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3373894701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:53.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:52 smithi111 ceph-mon[32784]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:41:53.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:52 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3842458307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:41:53.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:52 smithi111 ceph-mon[32784]: pgmap v287: 97 pgs: 97 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 548 B/s rd, 4.6 KiB/s wr, 20 op/s 2024-04-25T11:41:53.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:52 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3373894701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:41:53.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:53 smithi083 ceph-mon[29330]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:41:53.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:53 smithi083 ceph-mon[29330]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T11:41:53.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:53 smithi083 ceph-mon[29330]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:41:54.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:53 smithi094 ceph-mon[33032]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:41:54.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:53 smithi094 ceph-mon[33032]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T11:41:54.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:53 smithi094 ceph-mon[33032]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:41:54.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:53 smithi111 ceph-mon[32784]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:41:54.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:53 smithi111 ceph-mon[32784]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T11:41:54.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:53 smithi111 ceph-mon[32784]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:41:54.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.655+0000 7f88cd4c8700 1 -- 172.21.15.83:0/3723247485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8104690 msgr2=0x7f88c810af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:54.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.655+0000 7f88cd4c8700 1 --2- 172.21.15.83:0/3723247485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8104690 0x7f88c810af40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f88c000b0d0 tx=0x7f88c000b3e0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.655+0000 7f88cd4c8700 1 -- 172.21.15.83:0/3723247485 shutdown_connections 2024-04-25T11:41:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.655+0000 7f88cd4c8700 1 --2- 172.21.15.83:0/3723247485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8104690 0x7f88c810af40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.655+0000 7f88cd4c8700 1 --2- 172.21.15.83:0/3723247485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88c8102af0 0x7f88c8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.655+0000 7f88cd4c8700 1 --2- 172.21.15.83:0/3723247485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c8103cf0 0x7f88c8104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.655+0000 7f88cd4c8700 1 -- 172.21.15.83:0/3723247485 >> 172.21.15.83:0/3723247485 conn(0x7f88c80fe090 msgr2=0x7f88c81004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.655+0000 7f88cd4c8700 1 -- 172.21.15.83:0/3723247485 shutdown_connections 2024-04-25T11:41:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.655+0000 7f88cd4c8700 1 -- 172.21.15.83:0/3723247485 wait complete. 2024-04-25T11:41:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.655+0000 7f88cd4c8700 1 Processor -- start 2024-04-25T11:41:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88cd4c8700 1 -- start start 2024-04-25T11:41:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88cd4c8700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8102af0 0x7f88c8198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88cd4c8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c8103cf0 0x7f88c8199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88cd4c8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88c8104690 0x7f88c81a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88cd4c8700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88c810e680 con 0x7f88c8102af0 2024-04-25T11:41:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88cd4c8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88c810e500 con 0x7f88c8103cf0 2024-04-25T11:41:54.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88cd4c8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f88c810e800 con 0x7f88c8104690 2024-04-25T11:41:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88c6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8102af0 0x7f88c8198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88c6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8102af0 0x7f88c8198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46212/0 (socket says 172.21.15.83:46212) 2024-04-25T11:41:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88c6ffd700 1 -- 172.21.15.83:0/913941099 learned_addr learned my addr 172.21.15.83:0/913941099 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88be7fc700 1 --2- 172.21.15.83:0/913941099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c8103cf0 0x7f88c8199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88c6ffd700 1 -- 172.21.15.83:0/913941099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88c8104690 msgr2=0x7f88c81a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88c6ffd700 1 --2- 172.21.15.83:0/913941099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88c8104690 0x7f88c81a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88c6ffd700 1 -- 172.21.15.83:0/913941099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c8103cf0 msgr2=0x7f88c8199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88c6ffd700 1 --2- 172.21.15.83:0/913941099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c8103cf0 0x7f88c8199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88c6ffd700 1 -- 172.21.15.83:0/913941099 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c81a0870 con 0x7f88c8102af0 2024-04-25T11:41:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88be7fc700 1 --2- 172.21.15.83:0/913941099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c8103cf0 0x7f88c8199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:41:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88c6ffd700 1 --2- 172.21.15.83:0/913941099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8102af0 0x7f88c8198c20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88b800b500 tx=0x7f88b800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88c4ff9700 1 -- 172.21.15.83:0/913941099 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8015020 con 0x7f88c8102af0 2024-04-25T11:41:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.656+0000 7f88cd4c8700 1 -- 172.21.15.83:0/913941099 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c81a0b60 con 0x7f88c8102af0 2024-04-25T11:41:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.657+0000 7f88c4ff9700 1 -- 172.21.15.83:0/913941099 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b8011ce0 con 0x7f88c8102af0 2024-04-25T11:41:54.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.657+0000 7f88c4ff9700 1 -- 172.21.15.83:0/913941099 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b8010d60 con 0x7f88c8102af0 2024-04-25T11:41:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.657+0000 7f88cd4c8700 1 -- 172.21.15.83:0/913941099 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c81a10a0 con 0x7f88c8102af0 2024-04-25T11:41:54.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.658+0000 7f88cd4c8700 1 -- 172.21.15.83:0/913941099 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c8050f80 con 0x7f88c8102af0 2024-04-25T11:41:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.658+0000 7f88c4ff9700 1 -- 172.21.15.83:0/913941099 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f88b8018020 con 0x7f88c8102af0 2024-04-25T11:41:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.659+0000 7f88c4ff9700 1 --2- 172.21.15.83:0/913941099 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f88b40a3d80 0x7f88b40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.659+0000 7f88be7fc700 1 --2- 172.21.15.83:0/913941099 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f88b40a3d80 0x7f88b40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.659+0000 7f88c4ff9700 1 -- 172.21.15.83:0/913941099 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f88b80cfb60 con 0x7f88c8102af0 2024-04-25T11:41:54.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.659+0000 7f88be7fc700 1 --2- 172.21.15.83:0/913941099 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f88b40a3d80 0x7f88b40a6240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f88b0007900 tx=0x7f88b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:54.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.664+0000 7f88c4ff9700 1 -- 172.21.15.83:0/913941099 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88b801f020 con 0x7f88c8102af0 2024-04-25T11:41:54.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:54 smithi083 ceph-mon[29330]: pgmap v288: 97 pgs: 97 active+clean; 619 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 9.3 KiB/s wr, 24 op/s 2024-04-25T11:41:54.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.946+0000 7f88cd4c8700 1 -- 172.21.15.83:0/913941099 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f88c8063650 con 0x7f88c8102af0 2024-04-25T11:41:54.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.948+0000 7f88c4ff9700 1 -- 172.21.15.83:0/913941099 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) ==== 170+0+59 (secure 0 0 0) 0x7f88b809c6a0 con 0x7f88c8102af0 2024-04-25T11:41:54.935 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-25T11:41:54.935 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQCCQSpm/vp+OBAA/7KKsTwa80PctjlQHEup5g== 2024-04-25T11:41:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.950+0000 7f88cd4c8700 1 -- 172.21.15.83:0/913941099 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f88b40a3d80 msgr2=0x7f88b40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.950+0000 7f88cd4c8700 1 --2- 172.21.15.83:0/913941099 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f88b40a3d80 0x7f88b40a6240 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f88b0007900 tx=0x7f88b0008040 comp rx=0 tx=0).stop 2024-04-25T11:41:54.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.951+0000 7f88cd4c8700 1 -- 172.21.15.83:0/913941099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8102af0 msgr2=0x7f88c8198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.951+0000 7f88cd4c8700 1 --2- 172.21.15.83:0/913941099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8102af0 0x7f88c8198c20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88b800b500 tx=0x7f88b800b8c0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.951+0000 7f88cd4c8700 1 -- 172.21.15.83:0/913941099 shutdown_connections 2024-04-25T11:41:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.951+0000 7f88cd4c8700 1 --2- 172.21.15.83:0/913941099 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f88b40a3d80 0x7f88b40a6240 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.951+0000 7f88cd4c8700 1 --2- 172.21.15.83:0/913941099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88c8104690 0x7f88c81a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.951+0000 7f88cd4c8700 1 --2- 172.21.15.83:0/913941099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88c8102af0 0x7f88c8198c20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.951+0000 7f88cd4c8700 1 --2- 172.21.15.83:0/913941099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c8103cf0 0x7f88c8199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.951+0000 7f88cd4c8700 1 -- 172.21.15.83:0/913941099 >> 172.21.15.83:0/913941099 conn(0x7f88c80fe090 msgr2=0x7f88c8106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.951+0000 7f88cd4c8700 1 -- 172.21.15.83:0/913941099 shutdown_connections 2024-04-25T11:41:54.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:54.951+0000 7f88cd4c8700 1 -- 172.21.15.83:0/913941099 wait complete. 2024-04-25T11:41:55.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:54 smithi094 ceph-mon[33032]: pgmap v288: 97 pgs: 97 active+clean; 619 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 9.3 KiB/s wr, 24 op/s 2024-04-25T11:41:55.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:54 smithi111 ceph-mon[32784]: pgmap v288: 97 pgs: 97 active+clean; 619 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 9.3 KiB/s wr, 24 op/s 2024-04-25T11:41:55.535 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:41:55.536 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T11:41:55.536 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T11:41:55.571 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T11:41:55.571 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T11:41:55.571 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T11:41:55.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:55 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/913941099' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T11:41:55.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:55 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/913941099' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T11:41:55.773 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:41:55 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b[63285]: 2024-04-25T11:41:55.424+0000 7fa072f29700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-25T11:41:55.846 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:41:55.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:55 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/913941099' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T11:41:55.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:55 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/913941099' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T11:41:56.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:55 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/913941099' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T11:41:56.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:55 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/913941099' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T11:41:56.905 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:56 smithi083 ceph-mon[29330]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-04-25T11:41:57.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:56 smithi094 ceph-mon[33032]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-04-25T11:41:57.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:56 smithi111 ceph-mon[32784]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 11 KiB/s wr, 28 op/s 2024-04-25T11:41:57.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.551+0000 7f9cb8c9d700 1 -- 172.21.15.83:0/619730675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cb4102300 msgr2=0x7f9cb410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:57.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.551+0000 7f9cb8c9d700 1 --2- 172.21.15.83:0/619730675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cb4102300 0x7f9cb410e7f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00b0a0 tx=0x7f9cac00b3b0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.551+0000 7f9cb8c9d700 1 -- 172.21.15.83:0/619730675 shutdown_connections 2024-04-25T11:41:57.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.551+0000 7f9cb8c9d700 1 --2- 172.21.15.83:0/619730675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cb4102300 0x7f9cb410e7f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.551+0000 7f9cb8c9d700 1 --2- 172.21.15.83:0/619730675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb4107990 0x7f9cb4107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.551+0000 7f9cb8c9d700 1 --2- 172.21.15.83:0/619730675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb4101940 0x7f9cb4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.551+0000 7f9cb8c9d700 1 -- 172.21.15.83:0/619730675 >> 172.21.15.83:0/619730675 conn(0x7f9cb40788f0 msgr2=0x7f9cb4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:57.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.551+0000 7f9cb8c9d700 1 -- 172.21.15.83:0/619730675 shutdown_connections 2024-04-25T11:41:57.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.551+0000 7f9cb8c9d700 1 -- 172.21.15.83:0/619730675 wait complete. 2024-04-25T11:41:57.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.551+0000 7f9cb8c9d700 1 Processor -- start 2024-04-25T11:41:57.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.551+0000 7f9cb8c9d700 1 -- start start 2024-04-25T11:41:57.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb8c9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb4101940 0x7f9cb41990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:57.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb8c9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cb4102300 0x7f9cb4199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:57.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb8c9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb4107990 0x7f9cb419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:57.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb8c9d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb4112c80 con 0x7f9cb4102300 2024-04-25T11:41:57.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb8c9d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb4112b00 con 0x7f9cb4101940 2024-04-25T11:41:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb8c9d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb4112e00 con 0x7f9cb4107990 2024-04-25T11:41:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb1d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cb4102300 0x7f9cb4199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb1d9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cb4102300 0x7f9cb4199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46238/0 (socket says 172.21.15.83:46238) 2024-04-25T11:41:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb1d9b700 1 -- 172.21.15.83:0/3422647438 learned_addr learned my addr 172.21.15.83:0/3422647438 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:41:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb259c700 1 --2- 172.21.15.83:0/3422647438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb4101940 0x7f9cb41990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb2d9d700 1 --2- 172.21.15.83:0/3422647438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb4107990 0x7f9cb419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb1d9b700 1 -- 172.21.15.83:0/3422647438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb4107990 msgr2=0x7f9cb419d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb1d9b700 1 --2- 172.21.15.83:0/3422647438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb4107990 0x7f9cb419d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb1d9b700 1 -- 172.21.15.83:0/3422647438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb4101940 msgr2=0x7f9cb41990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb1d9b700 1 --2- 172.21.15.83:0/3422647438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb4101940 0x7f9cb41990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb1d9b700 1 -- 172.21.15.83:0/3422647438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb419e100 con 0x7f9cb4102300 2024-04-25T11:41:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb1d9b700 1 --2- 172.21.15.83:0/3422647438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cb4102300 0x7f9cb4199630 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800b530 tx=0x7f9ca800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:57.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9ca37fe700 1 -- 172.21.15.83:0/3422647438 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8015020 con 0x7f9cb4102300 2024-04-25T11:41:57.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9ca37fe700 1 -- 172.21.15.83:0/3422647438 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ca8011ce0 con 0x7f9cb4102300 2024-04-25T11:41:57.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9ca37fe700 1 -- 172.21.15.83:0/3422647438 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8010dd0 con 0x7f9cb4102300 2024-04-25T11:41:57.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.552+0000 7f9cb8c9d700 1 -- 172.21.15.83:0/3422647438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb419e3f0 con 0x7f9cb4102300 2024-04-25T11:41:57.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.553+0000 7f9cb8c9d700 1 -- 172.21.15.83:0/3422647438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb41a5c70 con 0x7f9cb4102300 2024-04-25T11:41:57.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.554+0000 7f9cb8c9d700 1 -- 172.21.15.83:0/3422647438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb4103060 con 0x7f9cb4102300 2024-04-25T11:41:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.555+0000 7f9ca37fe700 1 -- 172.21.15.83:0/3422647438 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9ca8018020 con 0x7f9cb4102300 2024-04-25T11:41:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.555+0000 7f9ca37fe700 1 --2- 172.21.15.83:0/3422647438 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9c9c0a3d50 0x7f9c9c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:41:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.555+0000 7f9cb259c700 1 --2- 172.21.15.83:0/3422647438 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9c9c0a3d50 0x7f9c9c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:41:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.555+0000 7f9ca37fe700 1 -- 172.21.15.83:0/3422647438 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9ca80cf920 con 0x7f9cb4102300 2024-04-25T11:41:57.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.556+0000 7f9cb259c700 1 --2- 172.21.15.83:0/3422647438 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9c9c0a3d50 0x7f9c9c0a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4005950 tx=0x7f9ca400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:41:57.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.560+0000 7f9ca37fe700 1 -- 172.21.15.83:0/3422647438 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ca801f020 con 0x7f9cb4102300 2024-04-25T11:41:57.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.816+0000 7f9cb8c9d700 1 -- 172.21.15.83:0/3422647438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f9cb402df80 con 0x7f9cb4102300 2024-04-25T11:41:57.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.821+0000 7f9ca37fe700 1 -- 172.21.15.83:0/3422647438 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7f9ca809c460 con 0x7f9cb4102300 2024-04-25T11:41:57.810 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:41:57.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.823+0000 7f9cb8c9d700 1 -- 172.21.15.83:0/3422647438 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9c9c0a3d50 msgr2=0x7f9c9c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:57.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.823+0000 7f9cb8c9d700 1 --2- 172.21.15.83:0/3422647438 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9c9c0a3d50 0x7f9c9c0a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4005950 tx=0x7f9ca400b410 comp rx=0 tx=0).stop 2024-04-25T11:41:57.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.823+0000 7f9cb8c9d700 1 -- 172.21.15.83:0/3422647438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cb4102300 msgr2=0x7f9cb4199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:41:57.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.823+0000 7f9cb8c9d700 1 --2- 172.21.15.83:0/3422647438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cb4102300 0x7f9cb4199630 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800b530 tx=0x7f9ca800b840 comp rx=0 tx=0).stop 2024-04-25T11:41:57.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.823+0000 7f9cb8c9d700 1 -- 172.21.15.83:0/3422647438 shutdown_connections 2024-04-25T11:41:57.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.823+0000 7f9cb8c9d700 1 --2- 172.21.15.83:0/3422647438 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9c9c0a3d50 0x7f9c9c0a6210 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.823+0000 7f9cb8c9d700 1 --2- 172.21.15.83:0/3422647438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb4107990 0x7f9cb419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.823+0000 7f9cb8c9d700 1 --2- 172.21.15.83:0/3422647438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb4101940 0x7f9cb41990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.823+0000 7f9cb8c9d700 1 --2- 172.21.15.83:0/3422647438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cb4102300 0x7f9cb4199630 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:41:57.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.823+0000 7f9cb8c9d700 1 -- 172.21.15.83:0/3422647438 >> 172.21.15.83:0/3422647438 conn(0x7f9cb40788f0 msgr2=0x7f9cb40ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:41:57.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.824+0000 7f9cb8c9d700 1 -- 172.21.15.83:0/3422647438 shutdown_connections 2024-04-25T11:41:57.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:41:57.824+0000 7f9cb8c9d700 1 -- 172.21.15.83:0/3422647438 wait complete. 2024-04-25T11:41:58.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:57 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3422647438' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T11:41:58.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:57 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3422647438' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T11:41:58.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:57 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3422647438' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T11:41:58.469 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":1157946519},{"type":"v1","addr":"172.21.15.83:6801","nonce":1157946519}]},"active_addr":"172.21.15.83:6801/1157946519","active_change":"2024-04-25T11:34:49.826221+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.83:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":457699907}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3642996010}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3934887058}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1837678414}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":2970918335}]}]} 2024-04-25T11:41:58.475 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T11:41:58.475 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T11:41:58.476 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T11:41:58.713 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:41:59.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:41:58 smithi083 ceph-mon[29330]: pgmap v290: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2024-04-25T11:41:59.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:41:58 smithi094 ceph-mon[33032]: pgmap v290: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2024-04-25T11:41:59.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:41:58 smithi111 ceph-mon[32784]: pgmap v290: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.8 KiB/s wr, 24 op/s 2024-04-25T11:42:00.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.341+0000 7fae02f5d700 1 -- 172.21.15.83:0/2981915672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadfc103450 msgr2=0x7fadfc10f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:00.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.341+0000 7fae02f5d700 1 --2- 172.21.15.83:0/2981915672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadfc103450 0x7fadfc10f8b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fadf400b0a0 tx=0x7fadf400b3b0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.341+0000 7fae02f5d700 1 -- 172.21.15.83:0/2981915672 shutdown_connections 2024-04-25T11:42:00.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.341+0000 7fae02f5d700 1 --2- 172.21.15.83:0/2981915672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadfc103450 0x7fadfc10f8b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.341+0000 7fae02f5d700 1 --2- 172.21.15.83:0/2981915672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadfc108a50 0x7fadfc108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.341+0000 7fae02f5d700 1 --2- 172.21.15.83:0/2981915672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadfc102a90 0x7fadfc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.341+0000 7fae02f5d700 1 -- 172.21.15.83:0/2981915672 >> 172.21.15.83:0/2981915672 conn(0x7fadfc079ba0 msgr2=0x7fadfc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:00.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.341+0000 7fae02f5d700 1 -- 172.21.15.83:0/2981915672 shutdown_connections 2024-04-25T11:42:00.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.341+0000 7fae02f5d700 1 -- 172.21.15.83:0/2981915672 wait complete. 2024-04-25T11:42:00.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae02f5d700 1 Processor -- start 2024-04-25T11:42:00.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae02f5d700 1 -- start start 2024-04-25T11:42:00.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae02f5d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadfc102a90 0x7fadfc1a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:00.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae02f5d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadfc103450 0x7fadfc1a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:00.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae02f5d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadfc108a50 0x7fadfc10eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:00.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae02f5d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc112d30 con 0x7fadfc102a90 2024-04-25T11:42:00.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae02f5d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc112bb0 con 0x7fadfc108a50 2024-04-25T11:42:00.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae02f5d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fadfc112eb0 con 0x7fadfc103450 2024-04-25T11:42:00.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae00cf9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadfc102a90 0x7fadfc1a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:00.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae00cf9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadfc102a90 0x7fadfc1a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46268/0 (socket says 172.21.15.83:46268) 2024-04-25T11:42:00.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae00cf9700 1 -- 172.21.15.83:0/3373457793 learned_addr learned my addr 172.21.15.83:0/3373457793 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:00.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae014fa700 1 --2- 172.21.15.83:0/3373457793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadfc108a50 0x7fadfc10eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:00.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fadfbfff700 1 --2- 172.21.15.83:0/3373457793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadfc103450 0x7fadfc1a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:00.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae00cf9700 1 -- 172.21.15.83:0/3373457793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadfc103450 msgr2=0x7fadfc1a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:00.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae00cf9700 1 --2- 172.21.15.83:0/3373457793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadfc103450 0x7fadfc1a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae00cf9700 1 -- 172.21.15.83:0/3373457793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadfc108a50 msgr2=0x7fadfc10eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:00.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae00cf9700 1 --2- 172.21.15.83:0/3373457793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadfc108a50 0x7fadfc10eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae00cf9700 1 -- 172.21.15.83:0/3373457793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadfc10f420 con 0x7fadfc102a90 2024-04-25T11:42:00.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.342+0000 7fae00cf9700 1 --2- 172.21.15.83:0/3373457793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadfc102a90 0x7fadfc1a5640 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fadf000b530 tx=0x7fadf000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:00.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.343+0000 7fadf9ffb700 1 -- 172.21.15.83:0/3373457793 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0015020 con 0x7fadfc102a90 2024-04-25T11:42:00.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.343+0000 7fadf9ffb700 1 -- 172.21.15.83:0/3373457793 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadf0011ce0 con 0x7fadfc102a90 2024-04-25T11:42:00.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.343+0000 7fae02f5d700 1 -- 172.21.15.83:0/3373457793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadfc10f710 con 0x7fadfc102a90 2024-04-25T11:42:00.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.343+0000 7fadf9ffb700 1 -- 172.21.15.83:0/3373457793 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0010e40 con 0x7fadfc102a90 2024-04-25T11:42:00.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.343+0000 7fae02f5d700 1 -- 172.21.15.83:0/3373457793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadfc1ac5c0 con 0x7fadfc102a90 2024-04-25T11:42:00.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.344+0000 7fae02f5d700 1 -- 172.21.15.83:0/3373457793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadfc104180 con 0x7fadfc102a90 2024-04-25T11:42:00.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.345+0000 7fadf9ffb700 1 -- 172.21.15.83:0/3373457793 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fadf0018020 con 0x7fadfc102a90 2024-04-25T11:42:00.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.346+0000 7fadf9ffb700 1 --2- 172.21.15.83:0/3373457793 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fadec0a3d50 0x7fadec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:00.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.346+0000 7fadfbfff700 1 --2- 172.21.15.83:0/3373457793 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fadec0a3d50 0x7fadec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:00.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.346+0000 7fadf9ffb700 1 -- 172.21.15.83:0/3373457793 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fadf00d0000 con 0x7fadfc102a90 2024-04-25T11:42:00.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.346+0000 7fadfbfff700 1 --2- 172.21.15.83:0/3373457793 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fadec0a3d50 0x7fadec0a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fade8006fd0 tx=0x7fade8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:00.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.350+0000 7fadf9ffb700 1 -- 172.21.15.83:0/3373457793 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fadf001f020 con 0x7fadfc102a90 2024-04-25T11:42:00.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.553+0000 7fae02f5d700 1 -- 172.21.15.83:0/3373457793 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadfc1a0c40 con 0x7fadfc102a90 2024-04-25T11:42:00.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.554+0000 7fadf9ffb700 1 -- 172.21.15.83:0/3373457793 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22147 (secure 0 0 0) 0x7fadf009cb40 con 0x7fadfc102a90 2024-04-25T11:42:00.541 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:42:00.542 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:41:45.888504+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:41:42.546991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:41:43.764226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:42:00.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.557+0000 7fae02f5d700 1 -- 172.21.15.83:0/3373457793 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fadec0a3d50 msgr2=0x7fadec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:00.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.557+0000 7fae02f5d700 1 --2- 172.21.15.83:0/3373457793 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fadec0a3d50 0x7fadec0a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fade8006fd0 tx=0x7fade8008040 comp rx=0 tx=0).stop 2024-04-25T11:42:00.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.557+0000 7fae02f5d700 1 -- 172.21.15.83:0/3373457793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadfc102a90 msgr2=0x7fadfc1a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:00.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.557+0000 7fae02f5d700 1 --2- 172.21.15.83:0/3373457793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadfc102a90 0x7fadfc1a5640 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fadf000b530 tx=0x7fadf000b840 comp rx=0 tx=0).stop 2024-04-25T11:42:00.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.557+0000 7fae02f5d700 1 -- 172.21.15.83:0/3373457793 shutdown_connections 2024-04-25T11:42:00.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.557+0000 7fae02f5d700 1 --2- 172.21.15.83:0/3373457793 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fadec0a3d50 0x7fadec0a6210 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.557+0000 7fae02f5d700 1 --2- 172.21.15.83:0/3373457793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadfc108a50 0x7fadfc10eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.557+0000 7fae02f5d700 1 --2- 172.21.15.83:0/3373457793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadfc102a90 0x7fadfc1a5640 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.557+0000 7fae02f5d700 1 --2- 172.21.15.83:0/3373457793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadfc103450 0x7fadfc1a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:00.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.557+0000 7fae02f5d700 1 -- 172.21.15.83:0/3373457793 >> 172.21.15.83:0/3373457793 conn(0x7fadfc079ba0 msgr2=0x7fadfc101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:00.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.557+0000 7fae02f5d700 1 -- 172.21.15.83:0/3373457793 shutdown_connections 2024-04-25T11:42:00.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:00.557+0000 7fae02f5d700 1 -- 172.21.15.83:0/3373457793 wait complete. 2024-04-25T11:42:01.216 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:00 smithi083 ceph-mon[29330]: pgmap v291: 97 pgs: 97 active+clean; 627 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.9 KiB/s wr, 27 op/s 2024-04-25T11:42:01.216 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:00 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3373457793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:42:01.217 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T11:42:01.218 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T11:42:01.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:00 smithi094 ceph-mon[33032]: pgmap v291: 97 pgs: 97 active+clean; 627 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.9 KiB/s wr, 27 op/s 2024-04-25T11:42:01.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:00 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3373457793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:42:01.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:00 smithi111 ceph-mon[32784]: pgmap v291: 97 pgs: 97 active+clean; 627 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 8.9 KiB/s wr, 27 op/s 2024-04-25T11:42:01.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:00 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3373457793' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:42:01.455 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.039+0000 7f284b513700 1 -- 172.21.15.83:0/263697693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2844103450 msgr2=0x7f284410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:03.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.039+0000 7f284b513700 1 --2- 172.21.15.83:0/263697693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2844103450 0x7f284410f8b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f283c00b0a0 tx=0x7f283c00b3b0 comp rx=0 tx=0).stop 2024-04-25T11:42:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.039+0000 7f284b513700 1 -- 172.21.15.83:0/263697693 shutdown_connections 2024-04-25T11:42:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.039+0000 7f284b513700 1 --2- 172.21.15.83:0/263697693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2844103450 0x7f284410f8b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.039+0000 7f284b513700 1 --2- 172.21.15.83:0/263697693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2844108a50 0x7f2844108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.039+0000 7f284b513700 1 --2- 172.21.15.83:0/263697693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2844102a90 0x7f2844102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.039+0000 7f284b513700 1 -- 172.21.15.83:0/263697693 >> 172.21.15.83:0/263697693 conn(0x7f2844079ba0 msgr2=0x7f2844079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.039+0000 7f284b513700 1 -- 172.21.15.83:0/263697693 shutdown_connections 2024-04-25T11:42:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.039+0000 7f284b513700 1 -- 172.21.15.83:0/263697693 wait complete. 2024-04-25T11:42:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f284b513700 1 Processor -- start 2024-04-25T11:42:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f284b513700 1 -- start start 2024-04-25T11:42:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f284b513700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2844102a90 0x7f2844199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f284b513700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2844103450 0x7f2844199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:03.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f284b513700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2844108a50 0x7f284419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f284b513700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2844112c00 con 0x7f2844102a90 2024-04-25T11:42:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f284b513700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2844112a80 con 0x7f2844103450 2024-04-25T11:42:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f284b513700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2844112d80 con 0x7f2844108a50 2024-04-25T11:42:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f28492af700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2844102a90 0x7f2844199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f28492af700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2844102a90 0x7f2844199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49120/0 (socket says 172.21.15.83:49120) 2024-04-25T11:42:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f28492af700 1 -- 172.21.15.83:0/3132067903 learned_addr learned my addr 172.21.15.83:0/3132067903 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f2849ab0700 1 --2- 172.21.15.83:0/3132067903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2844108a50 0x7f284419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f2848aae700 1 --2- 172.21.15.83:0/3132067903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2844103450 0x7f2844199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f28492af700 1 -- 172.21.15.83:0/3132067903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2844108a50 msgr2=0x7f284419da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f28492af700 1 --2- 172.21.15.83:0/3132067903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2844108a50 0x7f284419da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:03.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f28492af700 1 -- 172.21.15.83:0/3132067903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2844103450 msgr2=0x7f2844199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f28492af700 1 --2- 172.21.15.83:0/3132067903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2844103450 0x7f2844199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f28492af700 1 -- 172.21.15.83:0/3132067903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284419e140 con 0x7f2844102a90 2024-04-25T11:42:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.040+0000 7f2849ab0700 1 --2- 172.21.15.83:0/3132067903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2844108a50 0x7f284419da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:42:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.041+0000 7f28492af700 1 --2- 172.21.15.83:0/3132067903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2844102a90 0x7f2844199130 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f283400c970 tx=0x7f283400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.041+0000 7f283a7fc700 1 -- 172.21.15.83:0/3132067903 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834007aa0 con 0x7f2844102a90 2024-04-25T11:42:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.041+0000 7f283a7fc700 1 -- 172.21.15.83:0/3132067903 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2834004ce0 con 0x7f2844102a90 2024-04-25T11:42:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.041+0000 7f284b513700 1 -- 172.21.15.83:0/3132067903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284419e430 con 0x7f2844102a90 2024-04-25T11:42:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.041+0000 7f283a7fc700 1 -- 172.21.15.83:0/3132067903 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28340055c0 con 0x7f2844102a90 2024-04-25T11:42:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.041+0000 7f284b513700 1 -- 172.21.15.83:0/3132067903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28441a5cb0 con 0x7f2844102a90 2024-04-25T11:42:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.042+0000 7f284b513700 1 -- 172.21.15.83:0/3132067903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2844069730 con 0x7f2844102a90 2024-04-25T11:42:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.043+0000 7f283a7fc700 1 -- 172.21.15.83:0/3132067903 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2834007c40 con 0x7f2844102a90 2024-04-25T11:42:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.044+0000 7f283a7fc700 1 --2- 172.21.15.83:0/3132067903 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f28300a3d50 0x7f28300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:03.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.044+0000 7f2848aae700 1 --2- 172.21.15.83:0/3132067903 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f28300a3d50 0x7f28300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.044+0000 7f283a7fc700 1 -- 172.21.15.83:0/3132067903 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f28340d0540 con 0x7f2844102a90 2024-04-25T11:42:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.044+0000 7f2848aae700 1 --2- 172.21.15.83:0/3132067903 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f28300a3d50 0x7f28300a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2840005950 tx=0x7f28400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:03.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.048+0000 7f283a7fc700 1 -- 172.21.15.83:0/3132067903 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f283409cf90 con 0x7f2844102a90 2024-04-25T11:42:03.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:02 smithi083 ceph-mon[29330]: pgmap v292: 97 pgs: 97 active+clean; 627 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 22 op/s 2024-04-25T11:42:03.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.248+0000 7f284b513700 1 -- 172.21.15.83:0/3132067903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f284419a490 con 0x7f2844102a90 2024-04-25T11:42:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.249+0000 7f283a7fc700 1 -- 172.21.15.83:0/3132067903 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22147 (secure 0 0 0) 0x7f2834085e20 con 0x7f2844102a90 2024-04-25T11:42:03.236 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:42:03.236 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:41:45.888504+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:41:42.546991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:41:43.764226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:42:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.251+0000 7f284b513700 1 -- 172.21.15.83:0/3132067903 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f28300a3d50 msgr2=0x7f28300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.251+0000 7f284b513700 1 --2- 172.21.15.83:0/3132067903 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f28300a3d50 0x7f28300a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2840005950 tx=0x7f28400058e0 comp rx=0 tx=0).stop 2024-04-25T11:42:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.251+0000 7f284b513700 1 -- 172.21.15.83:0/3132067903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2844102a90 msgr2=0x7f2844199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.251+0000 7f284b513700 1 --2- 172.21.15.83:0/3132067903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2844102a90 0x7f2844199130 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f283400c970 tx=0x7f283400cd30 comp rx=0 tx=0).stop 2024-04-25T11:42:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.251+0000 7f284b513700 1 -- 172.21.15.83:0/3132067903 shutdown_connections 2024-04-25T11:42:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.251+0000 7f284b513700 1 --2- 172.21.15.83:0/3132067903 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f28300a3d50 0x7f28300a6210 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.251+0000 7f284b513700 1 --2- 172.21.15.83:0/3132067903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2844108a50 0x7f284419da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.251+0000 7f284b513700 1 --2- 172.21.15.83:0/3132067903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2844102a90 0x7f2844199130 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.251+0000 7f284b513700 1 --2- 172.21.15.83:0/3132067903 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2844103450 0x7f2844199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.251+0000 7f284b513700 1 -- 172.21.15.83:0/3132067903 >> 172.21.15.83:0/3132067903 conn(0x7f2844079ba0 msgr2=0x7f2844101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.251+0000 7f284b513700 1 -- 172.21.15.83:0/3132067903 shutdown_connections 2024-04-25T11:42:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:03.251+0000 7f284b513700 1 -- 172.21.15.83:0/3132067903 wait complete. 2024-04-25T11:42:03.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:02 smithi094 ceph-mon[33032]: pgmap v292: 97 pgs: 97 active+clean; 627 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 22 op/s 2024-04-25T11:42:03.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:02 smithi111 ceph-mon[32784]: pgmap v292: 97 pgs: 97 active+clean; 627 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 22 op/s 2024-04-25T11:42:03.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T11:42:03.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T11:42:03.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T11:42:03.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T11:42:03.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T11:42:03.903 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T11:42:03.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T11:42:03.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T11:42:03.905 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T11:42:03.905 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T11:42:03.905 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T11:42:03.906 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T11:42:04.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:03 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3132067903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:42:04.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:03 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3132067903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:42:04.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:03 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3132067903' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:42:04.559 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:04.658 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:04.987 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:04 smithi083 ceph-mon[29330]: pgmap v293: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-04-25T11:42:05.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:04 smithi094 ceph-mon[33032]: pgmap v293: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-04-25T11:42:05.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:04 smithi111 ceph-mon[32784]: pgmap v293: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.1 KiB/s wr, 18 op/s 2024-04-25T11:42:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:05.327 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:05.328 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:05.328 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:05.329 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:05.332 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:05.332 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:07.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:06 smithi094 ceph-mon[33032]: pgmap v294: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-04-25T11:42:07.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:06 smithi111 ceph-mon[32784]: pgmap v294: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-04-25T11:42:07.368 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:06 smithi083 ceph-mon[29330]: pgmap v294: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-04-25T11:42:08.702 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:09.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:08 smithi094 ceph-mon[33032]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-25T11:42:09.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:08 smithi083 ceph-mon[29330]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-25T11:42:09.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:08 smithi111 ceph-mon[32784]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-04-25T11:42:10.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.313+0000 7f9e1f732700 1 -- 172.21.15.83:0/8208138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1806b170 msgr2=0x7f9e1806b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.313+0000 7f9e1f732700 1 --2- 172.21.15.83:0/8208138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1806b170 0x7f9e1806b5d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9e080099c0 tx=0x7f9e08009cd0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.313+0000 7f9e1f732700 1 -- 172.21.15.83:0/8208138 shutdown_connections 2024-04-25T11:42:10.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.313+0000 7f9e1f732700 1 --2- 172.21.15.83:0/8208138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e1806bb10 0x7f9e1810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.313+0000 7f9e1f732700 1 --2- 172.21.15.83:0/8208138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e18108a30 0x7f9e18108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.313+0000 7f9e1f732700 1 --2- 172.21.15.83:0/8208138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1806b170 0x7f9e1806b5d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.313+0000 7f9e1f732700 1 -- 172.21.15.83:0/8208138 >> 172.21.15.83:0/8208138 conn(0x7f9e180788e0 msgr2=0x7f9e18078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:10.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.313+0000 7f9e1f732700 1 -- 172.21.15.83:0/8208138 shutdown_connections 2024-04-25T11:42:10.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.313+0000 7f9e1f732700 1 -- 172.21.15.83:0/8208138 wait complete. 2024-04-25T11:42:10.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.314+0000 7f9e1f732700 1 Processor -- start 2024-04-25T11:42:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.314+0000 7f9e1f732700 1 -- start start 2024-04-25T11:42:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.314+0000 7f9e1f732700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1806b170 0x7f9e18199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.314+0000 7f9e1f732700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e1806bb10 0x7f9e181996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.314+0000 7f9e1f732700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e18108a30 0x7f9e1819da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.314+0000 7f9e1f732700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9e18112c50 con 0x7f9e1806b170 2024-04-25T11:42:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.314+0000 7f9e1f732700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9e18112ad0 con 0x7f9e1806bb10 2024-04-25T11:42:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.314+0000 7f9e1f732700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9e18112dd0 con 0x7f9e18108a30 2024-04-25T11:42:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.314+0000 7f9e1d4ce700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1806b170 0x7f9e18199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.314+0000 7f9e1d4ce700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1806b170 0x7f9e18199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49140/0 (socket says 172.21.15.83:49140) 2024-04-25T11:42:10.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.314+0000 7f9e1d4ce700 1 -- 172.21.15.83:0/2677698770 learned_addr learned my addr 172.21.15.83:0/2677698770 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.315+0000 7f9e1cccd700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e1806bb10 0x7f9e181996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.315+0000 7f9e1dccf700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e18108a30 0x7f9e1819da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.315+0000 7f9e1d4ce700 1 -- 172.21.15.83:0/2677698770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e18108a30 msgr2=0x7f9e1819da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.315+0000 7f9e1d4ce700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e18108a30 0x7f9e1819da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.315+0000 7f9e1d4ce700 1 -- 172.21.15.83:0/2677698770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e1806bb10 msgr2=0x7f9e181996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.315+0000 7f9e1d4ce700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e1806bb10 0x7f9e181996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.315+0000 7f9e1d4ce700 1 -- 172.21.15.83:0/2677698770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1819e170 con 0x7f9e1806b170 2024-04-25T11:42:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.315+0000 7f9e1dccf700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e18108a30 0x7f9e1819da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.315+0000 7f9e1cccd700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e1806bb10 0x7f9e181996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:42:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.315+0000 7f9e1d4ce700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1806b170 0x7f9e18199160 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400b530 tx=0x7f9e1400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.315+0000 7f9e0e7fc700 1 -- 172.21.15.83:0/2677698770 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14015020 con 0x7f9e1806b170 2024-04-25T11:42:10.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.315+0000 7f9e0e7fc700 1 -- 172.21.15.83:0/2677698770 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e14011ce0 con 0x7f9e1806b170 2024-04-25T11:42:10.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.315+0000 7f9e0e7fc700 1 -- 172.21.15.83:0/2677698770 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14010d20 con 0x7f9e1806b170 2024-04-25T11:42:10.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.315+0000 7f9e1f732700 1 -- 172.21.15.83:0/2677698770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1819e460 con 0x7f9e1806b170 2024-04-25T11:42:10.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.315+0000 7f9e1f732700 1 -- 172.21.15.83:0/2677698770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e181a5ce0 con 0x7f9e1806b170 2024-04-25T11:42:10.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.316+0000 7f9e1f732700 1 -- 172.21.15.83:0/2677698770 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e180631d0 con 0x7f9e1806b170 2024-04-25T11:42:10.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.317+0000 7f9e0e7fc700 1 -- 172.21.15.83:0/2677698770 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9e14018020 con 0x7f9e1806b170 2024-04-25T11:42:10.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.318+0000 7f9e0e7fc700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9e040a3cd0 0x7f9e040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.318+0000 7f9e1cccd700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9e040a3cd0 0x7f9e040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.318+0000 7f9e0e7fc700 1 -- 172.21.15.83:0/2677698770 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9e140d0260 con 0x7f9e1806b170 2024-04-25T11:42:10.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.318+0000 7f9e0e7fc700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100] conn(0x7f9e040ad4b0 0x7f9e040af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:10.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.318+0000 7f9e0e7fc700 1 -- 172.21.15.83:0/2677698770 --> [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e14010ec0 con 0x7f9e040ad4b0 2024-04-25T11:42:10.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.318+0000 7f9e0e7fc700 1 -- 172.21.15.83:0/2677698770 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f9e140d0740 con 0x7f9e1806b170 2024-04-25T11:42:10.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.318+0000 7f9e1cccd700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9e040a3cd0 0x7f9e040a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9e18078fc0 tx=0x7f9e08005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:10.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.318+0000 7f9e1dccf700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100] conn(0x7f9e040ad4b0 0x7f9e040af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:10.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.318+0000 7f9e1dccf700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100] conn(0x7f9e040ad4b0 0x7f9e040af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:10.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.319+0000 7f9e0e7fc700 1 -- 172.21.15.83:0/2677698770 <== osd.9 v2:172.21.15.83:6826/3768581100 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e14010ec0 con 0x7f9e040ad4b0 2024-04-25T11:42:10.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.335+0000 7f9e1f732700 1 -- 172.21.15.83:0/2677698770 --> [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9e180635d0 con 0x7f9e040ad4b0 2024-04-25T11:42:10.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.336+0000 7f9e0e7fc700 1 -- 172.21.15.83:0/2677698770 <== osd.9 v2:172.21.15.83:6826/3768581100 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9e180635d0 con 0x7f9e040ad4b0 2024-04-25T11:42:10.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.336+0000 7f9e1f732700 1 -- 172.21.15.83:0/2677698770 >> [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100] conn(0x7f9e040ad4b0 msgr2=0x7f9e040af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.336+0000 7f9e1f732700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100] conn(0x7f9e040ad4b0 0x7f9e040af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.336+0000 7f9e1f732700 1 -- 172.21.15.83:0/2677698770 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9e040a3cd0 msgr2=0x7f9e040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.336+0000 7f9e1f732700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9e040a3cd0 0x7f9e040a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9e18078fc0 tx=0x7f9e08005fd0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.336+0000 7f9e1f732700 1 -- 172.21.15.83:0/2677698770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1806b170 msgr2=0x7f9e18199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:10.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.336+0000 7f9e1f732700 1 --2- 172.21.15.83:0/2677698770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e1806b170 0x7f9e18199160 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400b530 tx=0x7f9e1400b8f0 comp rx=0 tx=0).stop 2024-04-25T11:42:10.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.336+0000 7f9e1dccf700 1 -- 172.21.15.83:0/2677698770 reap_dead start 2024-04-25T11:42:10.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.336+0000 7f9e1f732700 1 -- 172.21.15.83:0/2677698770 shutdown_connections 2024-04-25T11:42:10.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.336+0000 7f9e1f732700 1 -- 172.21.15.83:0/2677698770 >> 172.21.15.83:0/2677698770 conn(0x7f9e180788e0 msgr2=0x7f9e180ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:10.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.336+0000 7f9e1f732700 1 -- 172.21.15.83:0/2677698770 shutdown_connections 2024-04-25T11:42:10.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:10.336+0000 7f9e1f732700 1 -- 172.21.15.83:0/2677698770 wait complete. 2024-04-25T11:42:11.228 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:10 smithi083 ceph-mon[29330]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-25T11:42:11.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:10 smithi094 ceph-mon[33032]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-25T11:42:11.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:10 smithi111 ceph-mon[32784]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-25T11:42:13.226 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:12 smithi083 ceph-mon[29330]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-25T11:42:13.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:12 smithi094 ceph-mon[33032]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-25T11:42:13.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:12 smithi111 ceph-mon[32784]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-25T11:42:13.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.635+0000 7f7c34b0d700 1 -- 172.21.15.83:0/1483528510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c30102d50 msgr2=0x7f7c301065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:13.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.635+0000 7f7c34b0d700 1 --2- 172.21.15.83:0/1483528510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c30102d50 0x7f7c301065e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7c180099c0 tx=0x7f7c18009cd0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.635+0000 7f7c34b0d700 1 -- 172.21.15.83:0/1483528510 shutdown_connections 2024-04-25T11:42:13.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.635+0000 7f7c34b0d700 1 --2- 172.21.15.83:0/1483528510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c30106cc0 0x7f7c3010e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.635+0000 7f7c34b0d700 1 --2- 172.21.15.83:0/1483528510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c301023a0 0x7f7c30102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.635+0000 7f7c34b0d700 1 --2- 172.21.15.83:0/1483528510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c30102d50 0x7f7c301065e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.635+0000 7f7c34b0d700 1 -- 172.21.15.83:0/1483528510 >> 172.21.15.83:0/1483528510 conn(0x7f7c30077fb0 msgr2=0x7f7c300ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:13.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.635+0000 7f7c34b0d700 1 -- 172.21.15.83:0/1483528510 shutdown_connections 2024-04-25T11:42:13.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.635+0000 7f7c34b0d700 1 -- 172.21.15.83:0/1483528510 wait complete. 2024-04-25T11:42:13.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.636+0000 7f7c34b0d700 1 Processor -- start 2024-04-25T11:42:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.636+0000 7f7c34b0d700 1 -- start start 2024-04-25T11:42:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.636+0000 7f7c34b0d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c301023a0 0x7f7c3019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.636+0000 7f7c34b0d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c30102d50 0x7f7c3019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.636+0000 7f7c34b0d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c30106cc0 0x7f7c301a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.636+0000 7f7c34b0d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c30111b70 con 0x7f7c30106cc0 2024-04-25T11:42:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.636+0000 7f7c34b0d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c301119f0 con 0x7f7c30102d50 2024-04-25T11:42:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.636+0000 7f7c34b0d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7c30111cf0 con 0x7f7c301023a0 2024-04-25T11:42:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.636+0000 7f7c2ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c30106cc0 0x7f7c301a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.636+0000 7f7c2ed9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c30106cc0 0x7f7c301a5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36120/0 (socket says 172.21.15.83:36120) 2024-04-25T11:42:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.636+0000 7f7c2ed9d700 1 -- 172.21.15.83:0/3928514654 learned_addr learned my addr 172.21.15.83:0/3928514654 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.637+0000 7f7c2e59c700 1 --2- 172.21.15.83:0/3928514654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c301023a0 0x7f7c3019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.637+0000 7f7c25d9b700 1 --2- 172.21.15.83:0/3928514654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c30102d50 0x7f7c3019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:13.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.637+0000 7f7c2ed9d700 1 -- 172.21.15.83:0/3928514654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c301023a0 msgr2=0x7f7c3019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:13.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.637+0000 7f7c2ed9d700 1 --2- 172.21.15.83:0/3928514654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c301023a0 0x7f7c3019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.637+0000 7f7c2ed9d700 1 -- 172.21.15.83:0/3928514654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c30102d50 msgr2=0x7f7c3019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:13.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.637+0000 7f7c2ed9d700 1 --2- 172.21.15.83:0/3928514654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c30102d50 0x7f7c3019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.637+0000 7f7c2ed9d700 1 -- 172.21.15.83:0/3928514654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c301a55d0 con 0x7f7c30106cc0 2024-04-25T11:42:13.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.637+0000 7f7c2ed9d700 1 --2- 172.21.15.83:0/3928514654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c30106cc0 0x7f7c301a5090 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7c28007d70 tx=0x7f7c2800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:13.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.637+0000 7f7c27fff700 1 -- 172.21.15.83:0/3928514654 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28016020 con 0x7f7c30106cc0 2024-04-25T11:42:13.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.637+0000 7f7c27fff700 1 -- 172.21.15.83:0/3928514654 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c2800ac90 con 0x7f7c30106cc0 2024-04-25T11:42:13.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.637+0000 7f7c34b0d700 1 -- 172.21.15.83:0/3928514654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c301a57a0 con 0x7f7c30106cc0 2024-04-25T11:42:13.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.637+0000 7f7c27fff700 1 -- 172.21.15.83:0/3928514654 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c280115e0 con 0x7f7c30106cc0 2024-04-25T11:42:13.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.637+0000 7f7c34b0d700 1 -- 172.21.15.83:0/3928514654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c301a5cb0 con 0x7f7c30106cc0 2024-04-25T11:42:13.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.638+0000 7f7c34b0d700 1 -- 172.21.15.83:0/3928514654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7c30063150 con 0x7f7c30106cc0 2024-04-25T11:42:13.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.639+0000 7f7c27fff700 1 -- 172.21.15.83:0/3928514654 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7c28019020 con 0x7f7c30106cc0 2024-04-25T11:42:13.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.640+0000 7f7c27fff700 1 --2- 172.21.15.83:0/3928514654 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c1c0a3cd0 0x7f7c1c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:13.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.640+0000 7f7c2e59c700 1 --2- 172.21.15.83:0/3928514654 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c1c0a3cd0 0x7f7c1c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:13.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.640+0000 7f7c27fff700 1 -- 172.21.15.83:0/3928514654 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f7c280d06d0 con 0x7f7c30106cc0 2024-04-25T11:42:13.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.640+0000 7f7c27fff700 1 --2- 172.21.15.83:0/3928514654 >> [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394] conn(0x7f7c1c0ad4b0 0x7f7c1c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:13.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.640+0000 7f7c27fff700 1 -- 172.21.15.83:0/3928514654 --> [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c28022e50 con 0x7f7c1c0ad4b0 2024-04-25T11:42:13.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.640+0000 7f7c27fff700 1 -- 172.21.15.83:0/3928514654 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f7c280d0bb0 con 0x7f7c30106cc0 2024-04-25T11:42:13.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.640+0000 7f7c2e59c700 1 --2- 172.21.15.83:0/3928514654 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c1c0a3cd0 0x7f7c1c0a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7c20005950 tx=0x7f7c2000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:13.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.640+0000 7f7c25d9b700 1 --2- 172.21.15.83:0/3928514654 >> [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394] conn(0x7f7c1c0ad4b0 0x7f7c1c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:13.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.641+0000 7f7c25d9b700 1 --2- 172.21.15.83:0/3928514654 >> [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394] conn(0x7f7c1c0ad4b0 0x7f7c1c0af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:13.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.642+0000 7f7c27fff700 1 -- 172.21.15.83:0/3928514654 <== osd.4 v2:172.21.15.94:6808/768239394 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7c28022e50 con 0x7f7c1c0ad4b0 2024-04-25T11:42:13.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.658+0000 7f7c34b0d700 1 -- 172.21.15.83:0/3928514654 --> [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7c30063550 con 0x7f7c1c0ad4b0 2024-04-25T11:42:13.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.659+0000 7f7c27fff700 1 -- 172.21.15.83:0/3928514654 <== osd.4 v2:172.21.15.94:6808/768239394 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7c30063550 con 0x7f7c1c0ad4b0 2024-04-25T11:42:13.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.659+0000 7f7c34b0d700 1 -- 172.21.15.83:0/3928514654 >> [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394] conn(0x7f7c1c0ad4b0 msgr2=0x7f7c1c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:13.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.659+0000 7f7c34b0d700 1 --2- 172.21.15.83:0/3928514654 >> [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394] conn(0x7f7c1c0ad4b0 0x7f7c1c0af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:13.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.659+0000 7f7c34b0d700 1 -- 172.21.15.83:0/3928514654 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c1c0a3cd0 msgr2=0x7f7c1c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:13.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.659+0000 7f7c34b0d700 1 --2- 172.21.15.83:0/3928514654 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c1c0a3cd0 0x7f7c1c0a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7c20005950 tx=0x7f7c2000a300 comp rx=0 tx=0).stop 2024-04-25T11:42:13.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.659+0000 7f7c34b0d700 1 -- 172.21.15.83:0/3928514654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c30106cc0 msgr2=0x7f7c301a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:13.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.659+0000 7f7c34b0d700 1 --2- 172.21.15.83:0/3928514654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c30106cc0 0x7f7c301a5090 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7c28007d70 tx=0x7f7c2800a340 comp rx=0 tx=0).stop 2024-04-25T11:42:13.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.659+0000 7f7c2ed9d700 1 -- 172.21.15.83:0/3928514654 reap_dead start 2024-04-25T11:42:13.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.659+0000 7f7c34b0d700 1 -- 172.21.15.83:0/3928514654 shutdown_connections 2024-04-25T11:42:13.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.659+0000 7f7c34b0d700 1 -- 172.21.15.83:0/3928514654 >> 172.21.15.83:0/3928514654 conn(0x7f7c30077fb0 msgr2=0x7f7c300ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:13.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.659+0000 7f7c34b0d700 1 -- 172.21.15.83:0/3928514654 shutdown_connections 2024-04-25T11:42:13.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:13.659+0000 7f7c34b0d700 1 -- 172.21.15.83:0/3928514654 wait complete. 2024-04-25T11:42:14.382 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:14.383 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:14.418 INFO:teuthology.orchestra.run.smithi083.stdout:146028888117 2024-04-25T11:42:14.418 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T11:42:15.043 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:14 smithi083 ceph-mon[29330]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-25T11:42:15.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:14 smithi094 ceph-mon[33032]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-25T11:42:15.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:14 smithi111 ceph-mon[32784]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-25T11:42:15.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.585+0000 7f42f120e700 1 -- 172.21.15.83:0/3584755278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42ec102300 msgr2=0x7f42ec10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.585+0000 7f42f120e700 1 --2- 172.21.15.83:0/3584755278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42ec102300 0x7f42ec10e7f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f42e400b0a0 tx=0x7f42e400b3b0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.585+0000 7f42f120e700 1 -- 172.21.15.83:0/3584755278 shutdown_connections 2024-04-25T11:42:15.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.585+0000 7f42f120e700 1 --2- 172.21.15.83:0/3584755278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42ec102300 0x7f42ec10e7f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.585+0000 7f42f120e700 1 --2- 172.21.15.83:0/3584755278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42ec107990 0x7f42ec107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.585+0000 7f42f120e700 1 --2- 172.21.15.83:0/3584755278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42ec101940 0x7f42ec101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.585+0000 7f42f120e700 1 -- 172.21.15.83:0/3584755278 >> 172.21.15.83:0/3584755278 conn(0x7f42ec0788f0 msgr2=0x7f42ec078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:15.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.585+0000 7f42f120e700 1 -- 172.21.15.83:0/3584755278 shutdown_connections 2024-04-25T11:42:15.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.585+0000 7f42f120e700 1 -- 172.21.15.83:0/3584755278 wait complete. 2024-04-25T11:42:15.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.586+0000 7f42f120e700 1 Processor -- start 2024-04-25T11:42:15.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.586+0000 7f42f120e700 1 -- start start 2024-04-25T11:42:15.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.586+0000 7f42f120e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42ec101940 0x7f42ec199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.586+0000 7f42f120e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42ec102300 0x7f42ec19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.586+0000 7f42f120e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42ec107990 0x7f42ec1a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.586+0000 7f42f120e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec111bb0 con 0x7f42ec102300 2024-04-25T11:42:15.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.586+0000 7f42f120e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec111a30 con 0x7f42ec101940 2024-04-25T11:42:15.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.586+0000 7f42f120e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec111d30 con 0x7f42ec107990 2024-04-25T11:42:15.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.586+0000 7f42ea59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42ec102300 0x7f42ec19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.586+0000 7f42ea59c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42ec102300 0x7f42ec19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36146/0 (socket says 172.21.15.83:36146) 2024-04-25T11:42:15.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.586+0000 7f42ea59c700 1 -- 172.21.15.83:0/1099248271 learned_addr learned my addr 172.21.15.83:0/1099248271 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:15.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.587+0000 7f42ead9d700 1 --2- 172.21.15.83:0/1099248271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42ec101940 0x7f42ec199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.587+0000 7f42eb59e700 1 --2- 172.21.15.83:0/1099248271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42ec107990 0x7f42ec1a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.587+0000 7f42ea59c700 1 -- 172.21.15.83:0/1099248271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42ec107990 msgr2=0x7f42ec1a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.587+0000 7f42ea59c700 1 --2- 172.21.15.83:0/1099248271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42ec107990 0x7f42ec1a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.587+0000 7f42ea59c700 1 -- 172.21.15.83:0/1099248271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42ec101940 msgr2=0x7f42ec199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.587+0000 7f42ea59c700 1 --2- 172.21.15.83:0/1099248271 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42ec101940 0x7f42ec199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.587+0000 7f42ea59c700 1 -- 172.21.15.83:0/1099248271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ec1a55b0 con 0x7f42ec102300 2024-04-25T11:42:15.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.587+0000 7f42ea59c700 1 --2- 172.21.15.83:0/1099248271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42ec102300 0x7f42ec19a530 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00c920 tx=0x7f42dc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:15.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.587+0000 7f42d3fff700 1 -- 172.21.15.83:0/1099248271 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc007b00 con 0x7f42ec102300 2024-04-25T11:42:15.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.587+0000 7f42f120e700 1 -- 172.21.15.83:0/1099248271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ec1a57e0 con 0x7f42ec102300 2024-04-25T11:42:15.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.587+0000 7f42d3fff700 1 -- 172.21.15.83:0/1099248271 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42dc004ce0 con 0x7f42ec102300 2024-04-25T11:42:15.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.587+0000 7f42d3fff700 1 -- 172.21.15.83:0/1099248271 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc005500 con 0x7f42ec102300 2024-04-25T11:42:15.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.587+0000 7f42f120e700 1 -- 172.21.15.83:0/1099248271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ec1a5d40 con 0x7f42ec102300 2024-04-25T11:42:15.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.588+0000 7f42f120e700 1 -- 172.21.15.83:0/1099248271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f42ec0630d0 con 0x7f42ec102300 2024-04-25T11:42:15.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.590+0000 7f42d3fff700 1 -- 172.21.15.83:0/1099248271 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f42dc007ca0 con 0x7f42ec102300 2024-04-25T11:42:15.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.590+0000 7f42d3fff700 1 --2- 172.21.15.83:0/1099248271 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42d40a3cd0 0x7f42d40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.590+0000 7f42ead9d700 1 --2- 172.21.15.83:0/1099248271 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42d40a3cd0 0x7f42d40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.591+0000 7f42d3fff700 1 -- 172.21.15.83:0/1099248271 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f42dc0d0340 con 0x7f42ec102300 2024-04-25T11:42:15.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.591+0000 7f42d3fff700 1 --2- 172.21.15.83:0/1099248271 >> [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106] conn(0x7f42d40ad4b0 0x7f42d40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.591+0000 7f42d3fff700 1 -- 172.21.15.83:0/1099248271 --> [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42dc012ec0 con 0x7f42d40ad4b0 2024-04-25T11:42:15.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.591+0000 7f42d3fff700 1 -- 172.21.15.83:0/1099248271 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f42dc0d0820 con 0x7f42ec102300 2024-04-25T11:42:15.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.591+0000 7f42eb59e700 1 --2- 172.21.15.83:0/1099248271 >> [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106] conn(0x7f42d40ad4b0 0x7f42d40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.591+0000 7f42ead9d700 1 --2- 172.21.15.83:0/1099248271 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42d40a3cd0 0x7f42d40a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009a10 tx=0x7f42e0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:15.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.591+0000 7f42eb59e700 1 --2- 172.21.15.83:0/1099248271 >> [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106] conn(0x7f42d40ad4b0 0x7f42d40af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:15.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.592+0000 7f42d3fff700 1 -- 172.21.15.83:0/1099248271 <== osd.0 v2:172.21.15.83:6802/2168030106 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42dc012ec0 con 0x7f42d40ad4b0 2024-04-25T11:42:15.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.608+0000 7f42f120e700 1 -- 172.21.15.83:0/1099248271 --> [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f42ec063550 con 0x7f42d40ad4b0 2024-04-25T11:42:15.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.608+0000 7f42d3fff700 1 -- 172.21.15.83:0/1099248271 <== osd.0 v2:172.21.15.83:6802/2168030106 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f42ec063550 con 0x7f42d40ad4b0 2024-04-25T11:42:15.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.609+0000 7f42f120e700 1 -- 172.21.15.83:0/1099248271 >> [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106] conn(0x7f42d40ad4b0 msgr2=0x7f42d40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.609+0000 7f42f120e700 1 --2- 172.21.15.83:0/1099248271 >> [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106] conn(0x7f42d40ad4b0 0x7f42d40af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.609+0000 7f42f120e700 1 -- 172.21.15.83:0/1099248271 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42d40a3cd0 msgr2=0x7f42d40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.609+0000 7f42f120e700 1 --2- 172.21.15.83:0/1099248271 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42d40a3cd0 0x7f42d40a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009a10 tx=0x7f42e0009190 comp rx=0 tx=0).stop 2024-04-25T11:42:15.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.609+0000 7f42f120e700 1 -- 172.21.15.83:0/1099248271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42ec102300 msgr2=0x7f42ec19a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.609+0000 7f42f120e700 1 --2- 172.21.15.83:0/1099248271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42ec102300 0x7f42ec19a530 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00c920 tx=0x7f42dc00cc30 comp rx=0 tx=0).stop 2024-04-25T11:42:15.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.609+0000 7f42eb59e700 1 -- 172.21.15.83:0/1099248271 reap_dead start 2024-04-25T11:42:15.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.609+0000 7f42f120e700 1 -- 172.21.15.83:0/1099248271 shutdown_connections 2024-04-25T11:42:15.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.609+0000 7f42f120e700 1 -- 172.21.15.83:0/1099248271 >> 172.21.15.83:0/1099248271 conn(0x7f42ec0788f0 msgr2=0x7f42ec0ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:15.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.609+0000 7f42f120e700 1 -- 172.21.15.83:0/1099248271 shutdown_connections 2024-04-25T11:42:15.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.609+0000 7f42f120e700 1 -- 172.21.15.83:0/1099248271 wait complete. 2024-04-25T11:42:15.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.733+0000 7f4ac84fd700 1 -- 172.21.15.83:0/178942239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac0101940 msgr2=0x7f4ac0101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.733+0000 7f4ac84fd700 1 --2- 172.21.15.83:0/178942239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac0101940 0x7f4ac0101dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009990 tx=0x7f4abc009ca0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.733+0000 7f4ac84fd700 1 -- 172.21.15.83:0/178942239 shutdown_connections 2024-04-25T11:42:15.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.733+0000 7f4ac84fd700 1 --2- 172.21.15.83:0/178942239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac0102300 0x7f4ac010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.733+0000 7f4ac84fd700 1 --2- 172.21.15.83:0/178942239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ac0107990 0x7f4ac0107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.733+0000 7f4ac84fd700 1 --2- 172.21.15.83:0/178942239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac0101940 0x7f4ac0101dc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.733+0000 7f4ac84fd700 1 -- 172.21.15.83:0/178942239 >> 172.21.15.83:0/178942239 conn(0x7f4ac00788f0 msgr2=0x7f4ac0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:15.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.733+0000 7f4ac84fd700 1 -- 172.21.15.83:0/178942239 shutdown_connections 2024-04-25T11:42:15.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.733+0000 7f4ac84fd700 1 -- 172.21.15.83:0/178942239 wait complete. 2024-04-25T11:42:15.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.733+0000 7f4ac84fd700 1 Processor -- start 2024-04-25T11:42:15.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac84fd700 1 -- start start 2024-04-25T11:42:15.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac84fd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac0101940 0x7f4ac0197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac84fd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ac0102300 0x7f4ac01982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac84fd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac0107990 0x7f4ac01987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac84fd700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0111b10 con 0x7f4ac0107990 2024-04-25T11:42:15.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac84fd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0111990 con 0x7f4ac0102300 2024-04-25T11:42:15.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac84fd700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0111c90 con 0x7f4ac0101940 2024-04-25T11:42:15.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac6a9a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac0107990 0x7f4ac01987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac6a9a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac0107990 0x7f4ac01987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36162/0 (socket says 172.21.15.83:36162) 2024-04-25T11:42:15.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac6a9a700 1 -- 172.21.15.83:0/2711936101 learned_addr learned my addr 172.21.15.83:0/2711936101 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:15.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac5a98700 1 --2- 172.21.15.83:0/2711936101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ac0102300 0x7f4ac01982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac6299700 1 --2- 172.21.15.83:0/2711936101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac0101940 0x7f4ac0197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac6a9a700 1 -- 172.21.15.83:0/2711936101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac0101940 msgr2=0x7f4ac0197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac6a9a700 1 --2- 172.21.15.83:0/2711936101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac0101940 0x7f4ac0197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac6a9a700 1 -- 172.21.15.83:0/2711936101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ac0102300 msgr2=0x7f4ac01982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac6a9a700 1 --2- 172.21.15.83:0/2711936101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4ac0102300 0x7f4ac01982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac6a9a700 1 -- 172.21.15.83:0/2711936101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac01a7570 con 0x7f4ac0107990 2024-04-25T11:42:15.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac6a9a700 1 --2- 172.21.15.83:0/2711936101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac0107990 0x7f4ac01987e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800ada0 tx=0x7f4ab800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:15.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ab77fe700 1 -- 172.21.15.83:0/2711936101 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8017020 con 0x7f4ac0107990 2024-04-25T11:42:15.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ac84fd700 1 -- 172.21.15.83:0/2711936101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac01a7860 con 0x7f4ac0107990 2024-04-25T11:42:15.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ab77fe700 1 -- 172.21.15.83:0/2711936101 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ab800cd80 con 0x7f4ac0107990 2024-04-25T11:42:15.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.734+0000 7f4ab77fe700 1 -- 172.21.15.83:0/2711936101 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8005260 con 0x7f4ac0107990 2024-04-25T11:42:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.735+0000 7f4ac84fd700 1 -- 172.21.15.83:0/2711936101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac01a7d70 con 0x7f4ac0107990 2024-04-25T11:42:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.735+0000 7f4ac84fd700 1 -- 172.21.15.83:0/2711936101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ac00631d0 con 0x7f4ac0107990 2024-04-25T11:42:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.736+0000 7f4ab77fe700 1 -- 172.21.15.83:0/2711936101 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4ab800c8d0 con 0x7f4ac0107990 2024-04-25T11:42:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.737+0000 7f4ab77fe700 1 --2- 172.21.15.83:0/2711936101 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4aac0a3cd0 0x7f4aac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.737+0000 7f4ac6299700 1 --2- 172.21.15.83:0/2711936101 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4aac0a3cd0 0x7f4aac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.737+0000 7f4ab77fe700 1 -- 172.21.15.83:0/2711936101 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4ab80d04c0 con 0x7f4ac0107990 2024-04-25T11:42:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.737+0000 7f4ab77fe700 1 --2- 172.21.15.83:0/2711936101 >> [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773] conn(0x7f4aac0ad4b0 0x7f4aac0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.737+0000 7f4ab77fe700 1 -- 172.21.15.83:0/2711936101 --> [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ab8019e50 con 0x7f4aac0ad4b0 2024-04-25T11:42:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.737+0000 7f4ab77fe700 1 -- 172.21.15.83:0/2711936101 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f4ab80d09a0 con 0x7f4ac0107990 2024-04-25T11:42:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.737+0000 7f4ac5a98700 1 --2- 172.21.15.83:0/2711936101 >> [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773] conn(0x7f4aac0ad4b0 0x7f4aac0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.737+0000 7f4ac6299700 1 --2- 172.21.15.83:0/2711936101 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4aac0a3cd0 0x7f4aac0a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000abd0 tx=0x7f4ab000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.738+0000 7f4ac5a98700 1 --2- 172.21.15.83:0/2711936101 >> [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773] conn(0x7f4aac0ad4b0 0x7f4aac0af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.738+0000 7f4ab77fe700 1 -- 172.21.15.83:0/2711936101 <== osd.6 v2:172.21.15.83:6818/305831773 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ab8019e50 con 0x7f4aac0ad4b0 2024-04-25T11:42:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.754+0000 7f4ac84fd700 1 -- 172.21.15.83:0/2711936101 --> [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ac0063650 con 0x7f4aac0ad4b0 2024-04-25T11:42:15.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.755+0000 7f4ab77fe700 1 -- 172.21.15.83:0/2711936101 <== osd.6 v2:172.21.15.83:6818/305831773 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4ac0063650 con 0x7f4aac0ad4b0 2024-04-25T11:42:15.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.755+0000 7f4ac84fd700 1 -- 172.21.15.83:0/2711936101 >> [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773] conn(0x7f4aac0ad4b0 msgr2=0x7f4aac0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.755+0000 7f4ac84fd700 1 --2- 172.21.15.83:0/2711936101 >> [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773] conn(0x7f4aac0ad4b0 0x7f4aac0af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:15.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.755+0000 7f4ac84fd700 1 -- 172.21.15.83:0/2711936101 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4aac0a3cd0 msgr2=0x7f4aac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.755+0000 7f4ac84fd700 1 --2- 172.21.15.83:0/2711936101 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4aac0a3cd0 0x7f4aac0a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000abd0 tx=0x7f4ab000a380 comp rx=0 tx=0).stop 2024-04-25T11:42:15.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.755+0000 7f4ac84fd700 1 -- 172.21.15.83:0/2711936101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac0107990 msgr2=0x7f4ac01987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:15.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.755+0000 7f4ac84fd700 1 --2- 172.21.15.83:0/2711936101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac0107990 0x7f4ac01987e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800ada0 tx=0x7f4ab800c330 comp rx=0 tx=0).stop 2024-04-25T11:42:15.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.755+0000 7f4ac6a9a700 1 -- 172.21.15.83:0/2711936101 reap_dead start 2024-04-25T11:42:15.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.755+0000 7f4ac84fd700 1 -- 172.21.15.83:0/2711936101 shutdown_connections 2024-04-25T11:42:15.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.755+0000 7f4ac84fd700 1 -- 172.21.15.83:0/2711936101 >> 172.21.15.83:0/2711936101 conn(0x7f4ac00788f0 msgr2=0x7f4ac00ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:15.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.755+0000 7f4ac84fd700 1 -- 172.21.15.83:0/2711936101 shutdown_connections 2024-04-25T11:42:15.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:15.755+0000 7f4ac84fd700 1 -- 172.21.15.83:0/2711936101 wait complete. 2024-04-25T11:42:16.620 INFO:teuthology.orchestra.run.smithi083.stdout:201863462956 2024-04-25T11:42:16.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T11:42:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.090+0000 7f8eb9610700 1 -- 172.21.15.83:0/2631953987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eb4101970 msgr2=0x7f8eb4101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.090+0000 7f8eb9610700 1 --2- 172.21.15.83:0/2631953987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eb4101970 0x7f8eb4101df0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80099c0 tx=0x7f8ea8009cd0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.090+0000 7f8eb9610700 1 -- 172.21.15.83:0/2631953987 shutdown_connections 2024-04-25T11:42:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.090+0000 7f8eb9610700 1 --2- 172.21.15.83:0/2631953987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8eb4102330 0x7f8eb410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.090+0000 7f8eb9610700 1 --2- 172.21.15.83:0/2631953987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8eb41079c0 0x7f8eb4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.090+0000 7f8eb9610700 1 --2- 172.21.15.83:0/2631953987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eb4101970 0x7f8eb4101df0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.090+0000 7f8eb9610700 1 -- 172.21.15.83:0/2631953987 >> 172.21.15.83:0/2631953987 conn(0x7f8eb4078980 msgr2=0x7f8eb4078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.090+0000 7f8eb9610700 1 -- 172.21.15.83:0/2631953987 shutdown_connections 2024-04-25T11:42:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.090+0000 7f8eb9610700 1 -- 172.21.15.83:0/2631953987 wait complete. 2024-04-25T11:42:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.091+0000 7f8eb9610700 1 Processor -- start 2024-04-25T11:42:17.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.091+0000 7f8eb9610700 1 -- start start 2024-04-25T11:42:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.091+0000 7f8eb9610700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8eb4101970 0x7f8eb419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.091+0000 7f8eb9610700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8eb4102330 0x7f8eb419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.091+0000 7f8eb9610700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eb41079c0 0x7f8eb41a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.091+0000 7f8eb9610700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4111bc0 con 0x7f8eb41079c0 2024-04-25T11:42:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.091+0000 7f8eb9610700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4111a40 con 0x7f8eb4102330 2024-04-25T11:42:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.091+0000 7f8eb9610700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb4111d40 con 0x7f8eb4101970 2024-04-25T11:42:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.091+0000 7f8eb37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eb41079c0 0x7f8eb41a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.091+0000 7f8eb37fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eb41079c0 0x7f8eb41a50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36192/0 (socket says 172.21.15.83:36192) 2024-04-25T11:42:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.091+0000 7f8eb37fe700 1 -- 172.21.15.83:0/702691022 learned_addr learned my addr 172.21.15.83:0/702691022 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.092+0000 7f8eb27fc700 1 --2- 172.21.15.83:0/702691022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8eb4102330 0x7f8eb419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.092+0000 7f8eb2ffd700 1 --2- 172.21.15.83:0/702691022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8eb4101970 0x7f8eb419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.092+0000 7f8eb37fe700 1 -- 172.21.15.83:0/702691022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8eb4101970 msgr2=0x7f8eb419a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.092+0000 7f8eb37fe700 1 --2- 172.21.15.83:0/702691022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8eb4101970 0x7f8eb419a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.092+0000 7f8eb37fe700 1 -- 172.21.15.83:0/702691022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8eb4102330 msgr2=0x7f8eb419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.092+0000 7f8eb37fe700 1 --2- 172.21.15.83:0/702691022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8eb4102330 0x7f8eb419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.092+0000 7f8eb37fe700 1 -- 172.21.15.83:0/702691022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb41a5600 con 0x7f8eb41079c0 2024-04-25T11:42:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.092+0000 7f8eb27fc700 1 --2- 172.21.15.83:0/702691022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8eb4102330 0x7f8eb419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:42:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.092+0000 7f8eb37fe700 1 --2- 172.21.15.83:0/702691022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eb41079c0 0x7f8eb41a50c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00ada0 tx=0x7f8eac00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.092+0000 7f8e9bfff700 1 -- 172.21.15.83:0/702691022 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac017020 con 0x7f8eb41079c0 2024-04-25T11:42:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.092+0000 7f8e9bfff700 1 -- 172.21.15.83:0/702691022 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eac00ce30 con 0x7f8eb41079c0 2024-04-25T11:42:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.092+0000 7f8e9bfff700 1 -- 172.21.15.83:0/702691022 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0054a0 con 0x7f8eb41079c0 2024-04-25T11:42:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.092+0000 7f8eb9610700 1 -- 172.21.15.83:0/702691022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb41a57d0 con 0x7f8eb41079c0 2024-04-25T11:42:17.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.092+0000 7f8eb9610700 1 -- 172.21.15.83:0/702691022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb41a5d30 con 0x7f8eb41079c0 2024-04-25T11:42:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.093+0000 7f8e99ffb700 1 -- 172.21.15.83:0/702691022 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8eb4063150 con 0x7f8eb41079c0 2024-04-25T11:42:17.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.094+0000 7f8e9bfff700 1 -- 172.21.15.83:0/702691022 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8eac00c980 con 0x7f8eb41079c0 2024-04-25T11:42:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.095+0000 7f8e9bfff700 1 --2- 172.21.15.83:0/702691022 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8e9c0a3d50 0x7f8e9c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.095+0000 7f8eb2ffd700 1 --2- 172.21.15.83:0/702691022 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8e9c0a3d50 0x7f8e9c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.095+0000 7f8e9bfff700 1 -- 172.21.15.83:0/702691022 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f8eac0d0540 con 0x7f8eb41079c0 2024-04-25T11:42:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.095+0000 7f8e9bfff700 1 --2- 172.21.15.83:0/702691022 >> [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753] conn(0x7f8e9c0ad530 0x7f8e9c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.095+0000 7f8e9bfff700 1 -- 172.21.15.83:0/702691022 --> [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e9c0b0000 con 0x7f8e9c0ad530 2024-04-25T11:42:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.095+0000 7f8e9bfff700 1 -- 172.21.15.83:0/702691022 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f8eac0d0a20 con 0x7f8eb41079c0 2024-04-25T11:42:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.095+0000 7f8eb2ffd700 1 --2- 172.21.15.83:0/702691022 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8e9c0a3d50 0x7f8e9c0a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4006fd0 tx=0x7f8ea4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.095+0000 7f8eb27fc700 1 --2- 172.21.15.83:0/702691022 >> [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753] conn(0x7f8e9c0ad530 0x7f8e9c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.096+0000 7f8eb27fc700 1 --2- 172.21.15.83:0/702691022 >> [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753] conn(0x7f8e9c0ad530 0x7f8e9c0af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:17.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.097+0000 7f8e9bfff700 1 -- 172.21.15.83:0/702691022 <== osd.10 v2:172.21.15.94:6824/2135498753 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e9c0b0000 con 0x7f8e9c0ad530 2024-04-25T11:42:17.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.113+0000 7f8e99ffb700 1 -- 172.21.15.83:0/702691022 --> [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8eb40635d0 con 0x7f8e9c0ad530 2024-04-25T11:42:17.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.113+0000 7f8e9bfff700 1 -- 172.21.15.83:0/702691022 <== osd.10 v2:172.21.15.94:6824/2135498753 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8eb40635d0 con 0x7f8e9c0ad530 2024-04-25T11:42:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.114+0000 7f8e99ffb700 1 -- 172.21.15.83:0/702691022 >> [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753] conn(0x7f8e9c0ad530 msgr2=0x7f8e9c0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.114+0000 7f8e99ffb700 1 --2- 172.21.15.83:0/702691022 >> [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753] conn(0x7f8e9c0ad530 0x7f8e9c0af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.114+0000 7f8e99ffb700 1 -- 172.21.15.83:0/702691022 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8e9c0a3d50 msgr2=0x7f8e9c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.114+0000 7f8e99ffb700 1 --2- 172.21.15.83:0/702691022 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8e9c0a3d50 0x7f8e9c0a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4006fd0 tx=0x7f8ea4008040 comp rx=0 tx=0).stop 2024-04-25T11:42:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.114+0000 7f8e99ffb700 1 -- 172.21.15.83:0/702691022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eb41079c0 msgr2=0x7f8eb41a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.114+0000 7f8e99ffb700 1 --2- 172.21.15.83:0/702691022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eb41079c0 0x7f8eb41a50c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00ada0 tx=0x7f8eac00c330 comp rx=0 tx=0).stop 2024-04-25T11:42:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.114+0000 7f8eb37fe700 1 -- 172.21.15.83:0/702691022 reap_dead start 2024-04-25T11:42:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.114+0000 7f8e99ffb700 1 -- 172.21.15.83:0/702691022 shutdown_connections 2024-04-25T11:42:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.114+0000 7f8e99ffb700 1 -- 172.21.15.83:0/702691022 >> 172.21.15.83:0/702691022 conn(0x7f8eb4078980 msgr2=0x7f8eb40ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.114+0000 7f8e99ffb700 1 -- 172.21.15.83:0/702691022 shutdown_connections 2024-04-25T11:42:17.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.114+0000 7f8e99ffb700 1 -- 172.21.15.83:0/702691022 wait complete. 2024-04-25T11:42:17.110 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:16 smithi083 ceph-mon[29330]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-25T11:42:17.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:16 smithi094 ceph-mon[33032]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-25T11:42:17.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:16 smithi111 ceph-mon[32784]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-25T11:42:17.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.296+0000 7f0059124700 1 -- 172.21.15.83:0/623809390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0054106950 msgr2=0x7f0054106d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.296+0000 7f0059124700 1 --2- 172.21.15.83:0/623809390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0054106950 0x7f0054106d30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0044009990 tx=0x7f0044009ca0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.296+0000 7f0059124700 1 -- 172.21.15.83:0/623809390 shutdown_connections 2024-04-25T11:42:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.296+0000 7f0059124700 1 --2- 172.21.15.83:0/623809390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0054102ac0 0x7f005410e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.296+0000 7f0059124700 1 --2- 172.21.15.83:0/623809390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0054106950 0x7f0054106d30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.296+0000 7f0059124700 1 --2- 172.21.15.83:0/623809390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054102100 0x7f0054102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.296+0000 7f0059124700 1 -- 172.21.15.83:0/623809390 >> 172.21.15.83:0/623809390 conn(0x7f0054077f40 msgr2=0x7f00540ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.296+0000 7f0059124700 1 -- 172.21.15.83:0/623809390 shutdown_connections 2024-04-25T11:42:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.296+0000 7f0059124700 1 -- 172.21.15.83:0/623809390 wait complete. 2024-04-25T11:42:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.297+0000 7f0059124700 1 Processor -- start 2024-04-25T11:42:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.297+0000 7f0059124700 1 -- start start 2024-04-25T11:42:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.297+0000 7f0059124700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0054102100 0x7f00541991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.297+0000 7f0059124700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0054102ac0 0x7f00541996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.297+0000 7f0059124700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054106950 0x7f005419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.297+0000 7f0059124700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0054111c60 con 0x7f0054102100 2024-04-25T11:42:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.297+0000 7f0059124700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0054111ae0 con 0x7f0054106950 2024-04-25T11:42:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.297+0000 7f0059124700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0054111de0 con 0x7f0054102ac0 2024-04-25T11:42:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.297+0000 7f0052d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0054102100 0x7f00541991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.297+0000 7f0052d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0054102100 0x7f00541991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36210/0 (socket says 172.21.15.83:36210) 2024-04-25T11:42:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.297+0000 7f0052d9d700 1 -- 172.21.15.83:0/3929894839 learned_addr learned my addr 172.21.15.83:0/3929894839 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.297+0000 7f005359e700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054106950 0x7f005419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.298+0000 7f005259c700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0054102ac0 0x7f00541996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.298+0000 7f0052d9d700 1 -- 172.21.15.83:0/3929894839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0054102ac0 msgr2=0x7f00541996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.298+0000 7f0052d9d700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0054102ac0 0x7f00541996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.298+0000 7f0052d9d700 1 -- 172.21.15.83:0/3929894839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054106950 msgr2=0x7f005419da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.298+0000 7f0052d9d700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054106950 0x7f005419da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.298+0000 7f0052d9d700 1 -- 172.21.15.83:0/3929894839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005419e170 con 0x7f0054102100 2024-04-25T11:42:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.298+0000 7f005359e700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054106950 0x7f005419da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:17.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.298+0000 7f005259c700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0054102ac0 0x7f00541996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.298+0000 7f0052d9d700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0054102100 0x7f00541991b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0044000c00 tx=0x7f00440047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.298+0000 7f003bfff700 1 -- 172.21.15.83:0/3929894839 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0044027070 con 0x7f0054102100 2024-04-25T11:42:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.298+0000 7f0059124700 1 -- 172.21.15.83:0/3929894839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005419e400 con 0x7f0054102100 2024-04-25T11:42:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.298+0000 7f003bfff700 1 -- 172.21.15.83:0/3929894839 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00440041d0 con 0x7f0054102100 2024-04-25T11:42:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.298+0000 7f003bfff700 1 -- 172.21.15.83:0/3929894839 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004400f460 con 0x7f0054102100 2024-04-25T11:42:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.298+0000 7f0059124700 1 -- 172.21.15.83:0/3929894839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00541a5ce0 con 0x7f0054102100 2024-04-25T11:42:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.299+0000 7f0059124700 1 -- 172.21.15.83:0/3929894839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0054063150 con 0x7f0054102100 2024-04-25T11:42:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.300+0000 7f003bfff700 1 -- 172.21.15.83:0/3929894839 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f004402a070 con 0x7f0054102100 2024-04-25T11:42:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.300+0000 7f003bfff700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f003c0a3cd0 0x7f003c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.301+0000 7f005259c700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f003c0a3cd0 0x7f003c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.301+0000 7f003bfff700 1 -- 172.21.15.83:0/3929894839 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f00440d1ab0 con 0x7f0054102100 2024-04-25T11:42:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.301+0000 7f003bfff700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209] conn(0x7f003c0ad4b0 0x7f003c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.301+0000 7f003bfff700 1 -- 172.21.15.83:0/3929894839 --> [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0044019e80 con 0x7f003c0ad4b0 2024-04-25T11:42:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.301+0000 7f003bfff700 1 -- 172.21.15.83:0/3929894839 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f00440d1f90 con 0x7f0054102100 2024-04-25T11:42:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.301+0000 7f005259c700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f003c0a3cd0 0x7f003c0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0048005fd0 tx=0x7f0048005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.301+0000 7f005359e700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209] conn(0x7f003c0ad4b0 0x7f003c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.301+0000 7f005359e700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209] conn(0x7f003c0ad4b0 0x7f003c0af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.302+0000 7f003bfff700 1 -- 172.21.15.83:0/3929894839 <== osd.3 v2:172.21.15.83:6810/775479209 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0044019e80 con 0x7f003c0ad4b0 2024-04-25T11:42:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.318+0000 7f0059124700 1 -- 172.21.15.83:0/3929894839 --> [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0054063550 con 0x7f003c0ad4b0 2024-04-25T11:42:17.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.318+0000 7f003bfff700 1 -- 172.21.15.83:0/3929894839 <== osd.3 v2:172.21.15.83:6810/775479209 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0054063550 con 0x7f003c0ad4b0 2024-04-25T11:42:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.319+0000 7f0059124700 1 -- 172.21.15.83:0/3929894839 >> [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209] conn(0x7f003c0ad4b0 msgr2=0x7f003c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.319+0000 7f0059124700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209] conn(0x7f003c0ad4b0 0x7f003c0af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.319+0000 7f0059124700 1 -- 172.21.15.83:0/3929894839 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f003c0a3cd0 msgr2=0x7f003c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.319+0000 7f0059124700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f003c0a3cd0 0x7f003c0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0048005fd0 tx=0x7f0048005e20 comp rx=0 tx=0).stop 2024-04-25T11:42:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.319+0000 7f0059124700 1 -- 172.21.15.83:0/3929894839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0054102100 msgr2=0x7f00541991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.319+0000 7f0059124700 1 --2- 172.21.15.83:0/3929894839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0054102100 0x7f00541991b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0044000c00 tx=0x7f00440047b0 comp rx=0 tx=0).stop 2024-04-25T11:42:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.319+0000 7f005359e700 1 -- 172.21.15.83:0/3929894839 reap_dead start 2024-04-25T11:42:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.319+0000 7f0059124700 1 -- 172.21.15.83:0/3929894839 shutdown_connections 2024-04-25T11:42:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.319+0000 7f0059124700 1 -- 172.21.15.83:0/3929894839 >> 172.21.15.83:0/3929894839 conn(0x7f0054077f40 msgr2=0x7f0054101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.319+0000 7f0059124700 1 -- 172.21.15.83:0/3929894839 shutdown_connections 2024-04-25T11:42:17.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:17.319+0000 7f0059124700 1 -- 172.21.15.83:0/3929894839 wait complete. 2024-04-25T11:42:18.114 INFO:teuthology.orchestra.run.smithi083.stdout:292057776157 2024-04-25T11:42:18.115 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T11:42:19.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:18 smithi094 ceph-mon[33032]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:42:19.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:18 smithi111 ceph-mon[32784]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:42:19.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:18 smithi083 ceph-mon[29330]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:42:19.478 INFO:teuthology.orchestra.run.smithi083.stdout:34359738440 2024-04-25T11:42:19.478 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T11:42:20.852 INFO:teuthology.orchestra.run.smithi083.stdout:313532612632 2024-04-25T11:42:20.853 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T11:42:21.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:20 smithi083 ceph-mon[29330]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T11:42:21.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:20 smithi094 ceph-mon[33032]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T11:42:21.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:20 smithi111 ceph-mon[32784]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T11:42:21.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.877+0000 7f54e033c700 1 -- 172.21.15.83:0/2518979451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54d8107940 msgr2=0x7f54d8107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:21.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.877+0000 7f54e033c700 1 --2- 172.21.15.83:0/2518979451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54d8107940 0x7f54d8107d20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009990 tx=0x7f54d4009ca0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.878+0000 7f54e033c700 1 -- 172.21.15.83:0/2518979451 shutdown_connections 2024-04-25T11:42:21.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.878+0000 7f54e033c700 1 --2- 172.21.15.83:0/2518979451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54d8102390 0x7f54d810e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.878+0000 7f54e033c700 1 --2- 172.21.15.83:0/2518979451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54d8107940 0x7f54d8107d20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.878+0000 7f54e033c700 1 --2- 172.21.15.83:0/2518979451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54d81019d0 0x7f54d8101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.878+0000 7f54e033c700 1 -- 172.21.15.83:0/2518979451 >> 172.21.15.83:0/2518979451 conn(0x7f54d8077990 msgr2=0x7f54d8077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:21.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.878+0000 7f54e033c700 1 -- 172.21.15.83:0/2518979451 shutdown_connections 2024-04-25T11:42:21.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.878+0000 7f54e033c700 1 -- 172.21.15.83:0/2518979451 wait complete. 2024-04-25T11:42:21.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.878+0000 7f54e033c700 1 Processor -- start 2024-04-25T11:42:21.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.878+0000 7f54e033c700 1 -- start start 2024-04-25T11:42:21.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.878+0000 7f54e033c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54d81019d0 0x7f54d8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:21.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.878+0000 7f54e033c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54d8102390 0x7f54d819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:21.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.878+0000 7f54e033c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54d8107940 0x7f54d81a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:21.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.878+0000 7f54e033c700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8111b40 con 0x7f54d8107940 2024-04-25T11:42:21.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.878+0000 7f54e033c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81119c0 con 0x7f54d8102390 2024-04-25T11:42:21.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.878+0000 7f54e033c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8111cc0 con 0x7f54d81019d0 2024-04-25T11:42:21.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54de8d9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54d8107940 0x7f54d81a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:21.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54de8d9700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54d8107940 0x7f54d81a5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36230/0 (socket says 172.21.15.83:36230) 2024-04-25T11:42:21.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54de8d9700 1 -- 172.21.15.83:0/2404042863 learned_addr learned my addr 172.21.15.83:0/2404042863 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:21.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54dd8d7700 1 --2- 172.21.15.83:0/2404042863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54d8102390 0x7f54d819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:21.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54de0d8700 1 --2- 172.21.15.83:0/2404042863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54d81019d0 0x7f54d8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:21.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54de8d9700 1 -- 172.21.15.83:0/2404042863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54d81019d0 msgr2=0x7f54d8199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:21.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54de8d9700 1 --2- 172.21.15.83:0/2404042863 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54d81019d0 0x7f54d8199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54de8d9700 1 -- 172.21.15.83:0/2404042863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54d8102390 msgr2=0x7f54d819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:21.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54de8d9700 1 --2- 172.21.15.83:0/2404042863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54d8102390 0x7f54d819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54de8d9700 1 -- 172.21.15.83:0/2404042863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d81a5560 con 0x7f54d8107940 2024-04-25T11:42:21.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54dd8d7700 1 --2- 172.21.15.83:0/2404042863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54d8102390 0x7f54d819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:42:21.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54de8d9700 1 --2- 172.21.15.83:0/2404042863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54d8107940 0x7f54d81a5020 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54d000ada0 tx=0x7f54d000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:21.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54cf7fe700 1 -- 172.21.15.83:0/2404042863 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0017020 con 0x7f54d8107940 2024-04-25T11:42:21.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54e033c700 1 -- 172.21.15.83:0/2404042863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d81a5790 con 0x7f54d8107940 2024-04-25T11:42:21.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54cf7fe700 1 -- 172.21.15.83:0/2404042863 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d000ce30 con 0x7f54d8107940 2024-04-25T11:42:21.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54cf7fe700 1 -- 172.21.15.83:0/2404042863 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d00054a0 con 0x7f54d8107940 2024-04-25T11:42:21.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.879+0000 7f54e033c700 1 -- 172.21.15.83:0/2404042863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d81a5ca0 con 0x7f54d8107940 2024-04-25T11:42:21.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.880+0000 7f54e033c700 1 -- 172.21.15.83:0/2404042863 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f54d8063150 con 0x7f54d8107940 2024-04-25T11:42:21.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.881+0000 7f54cf7fe700 1 -- 172.21.15.83:0/2404042863 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f54d000c980 con 0x7f54d8107940 2024-04-25T11:42:21.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.882+0000 7f54cf7fe700 1 --2- 172.21.15.83:0/2404042863 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f54c40a3cd0 0x7f54c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:21.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.882+0000 7f54de0d8700 1 --2- 172.21.15.83:0/2404042863 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f54c40a3cd0 0x7f54c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:21.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.882+0000 7f54cf7fe700 1 -- 172.21.15.83:0/2404042863 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f54d00d0530 con 0x7f54d8107940 2024-04-25T11:42:21.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.882+0000 7f54cf7fe700 1 --2- 172.21.15.83:0/2404042863 >> [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380] conn(0x7f54c40ad4b0 0x7f54c40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:21.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.882+0000 7f54cf7fe700 1 -- 172.21.15.83:0/2404042863 --> [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54d0019e50 con 0x7f54c40ad4b0 2024-04-25T11:42:21.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.882+0000 7f54cf7fe700 1 -- 172.21.15.83:0/2404042863 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f54d00d0a10 con 0x7f54d8107940 2024-04-25T11:42:21.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.882+0000 7f54de0d8700 1 --2- 172.21.15.83:0/2404042863 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f54c40a3cd0 0x7f54c40a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f54d4006010 tx=0x7f54d401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:21.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.882+0000 7f54dd8d7700 1 --2- 172.21.15.83:0/2404042863 >> [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380] conn(0x7f54c40ad4b0 0x7f54c40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:21.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.883+0000 7f54dd8d7700 1 --2- 172.21.15.83:0/2404042863 >> [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380] conn(0x7f54c40ad4b0 0x7f54c40af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:21.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.883+0000 7f54cf7fe700 1 -- 172.21.15.83:0/2404042863 <== osd.1 v2:172.21.15.94:6800/3582270380 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f54d0019e50 con 0x7f54c40ad4b0 2024-04-25T11:42:21.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.899+0000 7f54e033c700 1 -- 172.21.15.83:0/2404042863 --> [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f54d80635d0 con 0x7f54c40ad4b0 2024-04-25T11:42:21.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.899+0000 7f54cf7fe700 1 -- 172.21.15.83:0/2404042863 <== osd.1 v2:172.21.15.94:6800/3582270380 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f54d80635d0 con 0x7f54c40ad4b0 2024-04-25T11:42:21.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.900+0000 7f54e033c700 1 -- 172.21.15.83:0/2404042863 >> [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380] conn(0x7f54c40ad4b0 msgr2=0x7f54c40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:21.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.900+0000 7f54e033c700 1 --2- 172.21.15.83:0/2404042863 >> [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380] conn(0x7f54c40ad4b0 0x7f54c40af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:21.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.900+0000 7f54e033c700 1 -- 172.21.15.83:0/2404042863 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f54c40a3cd0 msgr2=0x7f54c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:21.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.900+0000 7f54e033c700 1 --2- 172.21.15.83:0/2404042863 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f54c40a3cd0 0x7f54c40a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f54d4006010 tx=0x7f54d401a040 comp rx=0 tx=0).stop 2024-04-25T11:42:21.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.900+0000 7f54e033c700 1 -- 172.21.15.83:0/2404042863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54d8107940 msgr2=0x7f54d81a5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:21.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.900+0000 7f54e033c700 1 --2- 172.21.15.83:0/2404042863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54d8107940 0x7f54d81a5020 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54d000ada0 tx=0x7f54d000c330 comp rx=0 tx=0).stop 2024-04-25T11:42:21.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.900+0000 7f54de8d9700 1 -- 172.21.15.83:0/2404042863 reap_dead start 2024-04-25T11:42:21.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.900+0000 7f54e033c700 1 -- 172.21.15.83:0/2404042863 shutdown_connections 2024-04-25T11:42:21.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.900+0000 7f54e033c700 1 -- 172.21.15.83:0/2404042863 >> 172.21.15.83:0/2404042863 conn(0x7f54d8077990 msgr2=0x7f54d80ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:21.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.900+0000 7f54e033c700 1 -- 172.21.15.83:0/2404042863 shutdown_connections 2024-04-25T11:42:21.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:21.900+0000 7f54e033c700 1 -- 172.21.15.83:0/2404042863 wait complete. 2024-04-25T11:42:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.307+0000 7f0d0388f700 1 -- 172.21.15.83:0/1699172521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cfc102ad0 msgr2=0x7f0cfc102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.307+0000 7f0d0388f700 1 --2- 172.21.15.83:0/1699172521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cfc102ad0 0x7f0cfc102f50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009990 tx=0x7f0cec009ca0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.307+0000 7f0d0388f700 1 -- 172.21.15.83:0/1699172521 shutdown_connections 2024-04-25T11:42:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.307+0000 7f0d0388f700 1 --2- 172.21.15.83:0/1699172521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cfc103490 0x7f0cfc10f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.307+0000 7f0d0388f700 1 --2- 172.21.15.83:0/1699172521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cfc108aa0 0x7f0cfc108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.307+0000 7f0d0388f700 1 --2- 172.21.15.83:0/1699172521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cfc102ad0 0x7f0cfc102f50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.307+0000 7f0d0388f700 1 -- 172.21.15.83:0/1699172521 >> 172.21.15.83:0/1699172521 conn(0x7f0cfc079c30 msgr2=0x7f0cfc07a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:22.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.307+0000 7f0d0388f700 1 -- 172.21.15.83:0/1699172521 shutdown_connections 2024-04-25T11:42:22.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.307+0000 7f0d0388f700 1 -- 172.21.15.83:0/1699172521 wait complete. 2024-04-25T11:42:22.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d0388f700 1 Processor -- start 2024-04-25T11:42:22.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d0388f700 1 -- start start 2024-04-25T11:42:22.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d0388f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cfc102ad0 0x7f0cfc074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:22.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d0388f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cfc103490 0x7f0cfc075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:22.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d0388f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cfc108aa0 0x7f0cfc0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:22.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d0388f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc112ce0 con 0x7f0cfc103490 2024-04-25T11:42:22.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d0388f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc112b60 con 0x7f0cfc108aa0 2024-04-25T11:42:22.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d0388f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc112e60 con 0x7f0cfc102ad0 2024-04-25T11:42:22.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d00e2a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cfc103490 0x7f0cfc075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:22.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d00e2a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cfc103490 0x7f0cfc075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36242/0 (socket says 172.21.15.83:36242) 2024-04-25T11:42:22.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d00e2a700 1 -- 172.21.15.83:0/3996621759 learned_addr learned my addr 172.21.15.83:0/3996621759 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:22.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d0162b700 1 --2- 172.21.15.83:0/3996621759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cfc102ad0 0x7f0cfc074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:22.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d01e2c700 1 --2- 172.21.15.83:0/3996621759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cfc108aa0 0x7f0cfc0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:22.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d00e2a700 1 -- 172.21.15.83:0/3996621759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cfc102ad0 msgr2=0x7f0cfc074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:22.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d00e2a700 1 --2- 172.21.15.83:0/3996621759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cfc102ad0 0x7f0cfc074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d00e2a700 1 -- 172.21.15.83:0/3996621759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cfc108aa0 msgr2=0x7f0cfc0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:22.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d00e2a700 1 --2- 172.21.15.83:0/3996621759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cfc108aa0 0x7f0cfc0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d00e2a700 1 -- 172.21.15.83:0/3996621759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc1a77a0 con 0x7f0cfc103490 2024-04-25T11:42:22.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d00e2a700 1 --2- 172.21.15.83:0/3996621759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cfc103490 0x7f0cfc075280 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0cec000c00 tx=0x7f0cec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:22.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0cf27fc700 1 -- 172.21.15.83:0/3996621759 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec027070 con 0x7f0cfc103490 2024-04-25T11:42:22.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d0388f700 1 -- 172.21.15.83:0/3996621759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cfc1a7a30 con 0x7f0cfc103490 2024-04-25T11:42:22.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0cf27fc700 1 -- 172.21.15.83:0/3996621759 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cec003fa0 con 0x7f0cfc103490 2024-04-25T11:42:22.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0cf27fc700 1 -- 172.21.15.83:0/3996621759 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cec022db0 con 0x7f0cfc103490 2024-04-25T11:42:22.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.308+0000 7f0d0388f700 1 -- 172.21.15.83:0/3996621759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cfc1a7e80 con 0x7f0cfc103490 2024-04-25T11:42:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.309+0000 7f0d0388f700 1 -- 172.21.15.83:0/3996621759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0cfc0630d0 con 0x7f0cfc103490 2024-04-25T11:42:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.311+0000 7f0cf27fc700 1 -- 172.21.15.83:0/3996621759 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0cec02a070 con 0x7f0cfc103490 2024-04-25T11:42:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.311+0000 7f0cf27fc700 1 --2- 172.21.15.83:0/3996621759 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0ce80a3cd0 0x7f0ce80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.312+0000 7f0d0162b700 1 --2- 172.21.15.83:0/3996621759 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0ce80a3cd0 0x7f0ce80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.312+0000 7f0cf27fc700 1 -- 172.21.15.83:0/3996621759 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0cec0d1710 con 0x7f0cfc103490 2024-04-25T11:42:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.312+0000 7f0cf27fc700 1 --2- 172.21.15.83:0/3996621759 >> [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490] conn(0x7f0ce80ad4b0 0x7f0ce80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.312+0000 7f0cf27fc700 1 -- 172.21.15.83:0/3996621759 --> [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0cec0043d0 con 0x7f0ce80ad4b0 2024-04-25T11:42:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.312+0000 7f0cf27fc700 1 -- 172.21.15.83:0/3996621759 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f0cec0d1bf0 con 0x7f0cfc103490 2024-04-25T11:42:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.312+0000 7f0d0162b700 1 --2- 172.21.15.83:0/3996621759 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0ce80a3cd0 0x7f0ce80a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800bbb0 tx=0x7f0cf800b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.312+0000 7f0d01e2c700 1 --2- 172.21.15.83:0/3996621759 >> [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490] conn(0x7f0ce80ad4b0 0x7f0ce80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.313+0000 7f0d01e2c700 1 --2- 172.21.15.83:0/3996621759 >> [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490] conn(0x7f0ce80ad4b0 0x7f0ce80af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.313+0000 7f0cf27fc700 1 -- 172.21.15.83:0/3996621759 <== osd.5 v2:172.21.15.111:6808/3244351490 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0cec0043d0 con 0x7f0ce80ad4b0 2024-04-25T11:42:22.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.329+0000 7f0d0388f700 1 -- 172.21.15.83:0/3996621759 --> [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0cfc0634d0 con 0x7f0ce80ad4b0 2024-04-25T11:42:22.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.330+0000 7f0cf27fc700 1 -- 172.21.15.83:0/3996621759 <== osd.5 v2:172.21.15.111:6808/3244351490 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0cfc0634d0 con 0x7f0ce80ad4b0 2024-04-25T11:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.330+0000 7f0d0388f700 1 -- 172.21.15.83:0/3996621759 >> [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490] conn(0x7f0ce80ad4b0 msgr2=0x7f0ce80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.330+0000 7f0d0388f700 1 --2- 172.21.15.83:0/3996621759 >> [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490] conn(0x7f0ce80ad4b0 0x7f0ce80af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.330+0000 7f0d0388f700 1 -- 172.21.15.83:0/3996621759 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0ce80a3cd0 msgr2=0x7f0ce80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.330+0000 7f0d0388f700 1 --2- 172.21.15.83:0/3996621759 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0ce80a3cd0 0x7f0ce80a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800bbb0 tx=0x7f0cf800b590 comp rx=0 tx=0).stop 2024-04-25T11:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.330+0000 7f0d0388f700 1 -- 172.21.15.83:0/3996621759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cfc103490 msgr2=0x7f0cfc075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.330+0000 7f0d0388f700 1 --2- 172.21.15.83:0/3996621759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cfc103490 0x7f0cfc075280 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0cec000c00 tx=0x7f0cec0047b0 comp rx=0 tx=0).stop 2024-04-25T11:42:22.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.330+0000 7f0d01e2c700 1 -- 172.21.15.83:0/3996621759 reap_dead start 2024-04-25T11:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.330+0000 7f0d0388f700 1 -- 172.21.15.83:0/3996621759 shutdown_connections 2024-04-25T11:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.330+0000 7f0d0388f700 1 -- 172.21.15.83:0/3996621759 >> 172.21.15.83:0/3996621759 conn(0x7f0cfc079c30 msgr2=0x7f0cfc100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.330+0000 7f0d0388f700 1 -- 172.21.15.83:0/3996621759 shutdown_connections 2024-04-25T11:42:22.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:22.331+0000 7f0d0388f700 1 -- 172.21.15.83:0/3996621759 wait complete. 2024-04-25T11:42:23.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:23 smithi083 ceph-mon[29330]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:42:23.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:23 smithi094 ceph-mon[33032]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:42:23.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:23 smithi111 ceph-mon[32784]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:42:23.582 INFO:teuthology.orchestra.run.smithi083.stdout:55834574916 2024-04-25T11:42:23.583 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T11:42:24.794 INFO:teuthology.orchestra.run.smithi083.stdout:111669149755 2024-04-25T11:42:24.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T11:42:25.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:25 smithi083 ceph-mon[29330]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T11:42:25.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:25 smithi094 ceph-mon[33032]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T11:42:25.317 INFO:teuthology.orchestra.run.smithi083.stdout:167503724593 2024-04-25T11:42:25.317 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T11:42:25.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:25 smithi111 ceph-mon[32784]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-25T11:42:25.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.955+0000 7fed86737700 1 -- 172.21.15.83:0/1897275972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed801019d0 msgr2=0x7fed8010f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:25.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.955+0000 7fed86737700 1 --2- 172.21.15.83:0/1897275972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed801019d0 0x7fed8010f910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fed7800b0a0 tx=0x7fed7800b3b0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.955+0000 7fed86737700 1 -- 172.21.15.83:0/1897275972 shutdown_connections 2024-04-25T11:42:25.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.955+0000 7fed86737700 1 --2- 172.21.15.83:0/1897275972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed801019d0 0x7fed8010f910 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.955+0000 7fed86737700 1 --2- 172.21.15.83:0/1897275972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed80108ad0 0x7fed80108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.955+0000 7fed86737700 1 --2- 172.21.15.83:0/1897275972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed80101030 0x7fed80101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.955+0000 7fed86737700 1 -- 172.21.15.83:0/1897275972 >> 172.21.15.83:0/1897275972 conn(0x7fed80077fb0 msgr2=0x7fed800ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.955+0000 7fed86737700 1 -- 172.21.15.83:0/1897275972 shutdown_connections 2024-04-25T11:42:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.955+0000 7fed86737700 1 -- 172.21.15.83:0/1897275972 wait complete. 2024-04-25T11:42:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.956+0000 7fed86737700 1 Processor -- start 2024-04-25T11:42:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.956+0000 7fed86737700 1 -- start start 2024-04-25T11:42:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.956+0000 7fed86737700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed80101030 0x7fed80102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.956+0000 7fed86737700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed801019d0 0x7fed80102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.956+0000 7fed86737700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed80108ad0 0x7fed80102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.956+0000 7fed84cd4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed80108ad0 0x7fed80102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.956+0000 7fed84cd4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed80108ad0 0x7fed80102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60058/0 (socket says 172.21.15.83:60058) 2024-04-25T11:42:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.956+0000 7fed84cd4700 1 -- 172.21.15.83:0/3301713874 learned_addr learned my addr 172.21.15.83:0/3301713874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.956+0000 7fed86737700 1 -- 172.21.15.83:0/3301713874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fed80112be0 con 0x7fed80108ad0 2024-04-25T11:42:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.956+0000 7fed86737700 1 -- 172.21.15.83:0/3301713874 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fed80112a60 con 0x7fed80101030 2024-04-25T11:42:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.957+0000 7fed86737700 1 -- 172.21.15.83:0/3301713874 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fed80112d60 con 0x7fed801019d0 2024-04-25T11:42:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.957+0000 7fed7ffff700 1 --2- 172.21.15.83:0/3301713874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed80101030 0x7fed80102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:25.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.957+0000 7fed7f7fe700 1 --2- 172.21.15.83:0/3301713874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed801019d0 0x7fed80102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:25.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.957+0000 7fed84cd4700 1 -- 172.21.15.83:0/3301713874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed801019d0 msgr2=0x7fed80102750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:25.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.957+0000 7fed84cd4700 1 --2- 172.21.15.83:0/3301713874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed801019d0 0x7fed80102750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.957+0000 7fed84cd4700 1 -- 172.21.15.83:0/3301713874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed80101030 msgr2=0x7fed80102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:25.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.957+0000 7fed84cd4700 1 --2- 172.21.15.83:0/3301713874 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fed80101030 0x7fed80102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.957+0000 7fed84cd4700 1 -- 172.21.15.83:0/3301713874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed801a75e0 con 0x7fed80108ad0 2024-04-25T11:42:25.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.957+0000 7fed84cd4700 1 --2- 172.21.15.83:0/3301713874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed80108ad0 0x7fed80102c90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fed780062a0 tx=0x7fed78004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:25.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.957+0000 7fed7d7fa700 1 -- 172.21.15.83:0/3301713874 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78021070 con 0x7fed80108ad0 2024-04-25T11:42:25.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.957+0000 7fed86737700 1 -- 172.21.15.83:0/3301713874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed801a7870 con 0x7fed80108ad0 2024-04-25T11:42:25.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.957+0000 7fed7d7fa700 1 -- 172.21.15.83:0/3301713874 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed78007c80 con 0x7fed80108ad0 2024-04-25T11:42:25.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.957+0000 7fed7d7fa700 1 -- 172.21.15.83:0/3301713874 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7801d450 con 0x7fed80108ad0 2024-04-25T11:42:25.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.957+0000 7fed86737700 1 -- 172.21.15.83:0/3301713874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed801a7d70 con 0x7fed80108ad0 2024-04-25T11:42:25.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.958+0000 7fed86737700 1 -- 172.21.15.83:0/3301713874 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed80063150 con 0x7fed80108ad0 2024-04-25T11:42:25.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.959+0000 7fed7d7fa700 1 -- 172.21.15.83:0/3301713874 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fed78004000 con 0x7fed80108ad0 2024-04-25T11:42:25.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.960+0000 7fed7d7fa700 1 --2- 172.21.15.83:0/3301713874 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fed680a3d50 0x7fed680a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:25.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.960+0000 7fed7ffff700 1 --2- 172.21.15.83:0/3301713874 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fed680a3d50 0x7fed680a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:25.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.960+0000 7fed7d7fa700 1 -- 172.21.15.83:0/3301713874 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fed7800f020 con 0x7fed80108ad0 2024-04-25T11:42:25.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.960+0000 7fed7d7fa700 1 --2- 172.21.15.83:0/3301713874 >> [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463] conn(0x7fed680ad530 0x7fed680af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:25.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.960+0000 7fed7d7fa700 1 -- 172.21.15.83:0/3301713874 --> [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed7802ee50 con 0x7fed680ad530 2024-04-25T11:42:25.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.960+0000 7fed7d7fa700 1 -- 172.21.15.83:0/3301713874 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fed780d2140 con 0x7fed80108ad0 2024-04-25T11:42:25.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.960+0000 7fed7f7fe700 1 --2- 172.21.15.83:0/3301713874 >> [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463] conn(0x7fed680ad530 0x7fed680af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:25.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.960+0000 7fed7ffff700 1 --2- 172.21.15.83:0/3301713874 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fed680a3d50 0x7fed680a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fed70005950 tx=0x7fed7000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:25.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.961+0000 7fed7f7fe700 1 --2- 172.21.15.83:0/3301713874 >> [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463] conn(0x7fed680ad530 0x7fed680af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:25.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.962+0000 7fed7d7fa700 1 -- 172.21.15.83:0/3301713874 <== osd.7 v2:172.21.15.94:6816/4200260463 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed7802ee50 con 0x7fed680ad530 2024-04-25T11:42:25.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.978+0000 7fed86737700 1 -- 172.21.15.83:0/3301713874 --> [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fed80063550 con 0x7fed680ad530 2024-04-25T11:42:25.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.979+0000 7fed7d7fa700 1 -- 172.21.15.83:0/3301713874 <== osd.7 v2:172.21.15.94:6816/4200260463 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fed80063550 con 0x7fed680ad530 2024-04-25T11:42:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.979+0000 7fed86737700 1 -- 172.21.15.83:0/3301713874 >> [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463] conn(0x7fed680ad530 msgr2=0x7fed680af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.979+0000 7fed86737700 1 --2- 172.21.15.83:0/3301713874 >> [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463] conn(0x7fed680ad530 0x7fed680af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.979+0000 7fed86737700 1 -- 172.21.15.83:0/3301713874 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fed680a3d50 msgr2=0x7fed680a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.979+0000 7fed86737700 1 --2- 172.21.15.83:0/3301713874 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fed680a3d50 0x7fed680a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fed70005950 tx=0x7fed7000a300 comp rx=0 tx=0).stop 2024-04-25T11:42:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.979+0000 7fed86737700 1 -- 172.21.15.83:0/3301713874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed80108ad0 msgr2=0x7fed80102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.979+0000 7fed86737700 1 --2- 172.21.15.83:0/3301713874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fed80108ad0 0x7fed80102c90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fed780062a0 tx=0x7fed78004e20 comp rx=0 tx=0).stop 2024-04-25T11:42:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.979+0000 7fed84cd4700 1 -- 172.21.15.83:0/3301713874 reap_dead start 2024-04-25T11:42:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.979+0000 7fed86737700 1 -- 172.21.15.83:0/3301713874 shutdown_connections 2024-04-25T11:42:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.979+0000 7fed86737700 1 -- 172.21.15.83:0/3301713874 >> 172.21.15.83:0/3301713874 conn(0x7fed80077fb0 msgr2=0x7fed800ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.979+0000 7fed86737700 1 -- 172.21.15.83:0/3301713874 shutdown_connections 2024-04-25T11:42:25.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:25.979+0000 7fed86737700 1 -- 172.21.15.83:0/3301713874 wait complete. 2024-04-25T11:42:26.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.984+0000 7f69faa1b700 1 -- 172.21.15.83:0/3456982106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f4103450 msgr2=0x7f69f410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.984+0000 7f69faa1b700 1 --2- 172.21.15.83:0/3456982106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f4103450 0x7f69f410f8b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00b0a0 tx=0x7f69ec00b3b0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.984+0000 7f69faa1b700 1 -- 172.21.15.83:0/3456982106 shutdown_connections 2024-04-25T11:42:26.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.984+0000 7f69faa1b700 1 --2- 172.21.15.83:0/3456982106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f4103450 0x7f69f410f8b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.984+0000 7f69faa1b700 1 --2- 172.21.15.83:0/3456982106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f4108a50 0x7f69f4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.984+0000 7f69faa1b700 1 --2- 172.21.15.83:0/3456982106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69f4102a90 0x7f69f4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.984+0000 7f69faa1b700 1 -- 172.21.15.83:0/3456982106 >> 172.21.15.83:0/3456982106 conn(0x7f69f4079ba0 msgr2=0x7f69f4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:26.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.984+0000 7f69faa1b700 1 -- 172.21.15.83:0/3456982106 shutdown_connections 2024-04-25T11:42:26.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.984+0000 7f69faa1b700 1 -- 172.21.15.83:0/3456982106 wait complete. 2024-04-25T11:42:26.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.985+0000 7f69faa1b700 1 Processor -- start 2024-04-25T11:42:26.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.985+0000 7f69faa1b700 1 -- start start 2024-04-25T11:42:26.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.985+0000 7f69faa1b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f4102a90 0x7f69f4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.985+0000 7f69faa1b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69f4103450 0x7f69f4075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.985+0000 7f69faa1b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f4108a50 0x7f69f40757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.985+0000 7f69faa1b700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f69f4112cb0 con 0x7f69f4102a90 2024-04-25T11:42:26.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.985+0000 7f69faa1b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69f4112b30 con 0x7f69f4108a50 2024-04-25T11:42:26.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.985+0000 7f69faa1b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f69f4112e30 con 0x7f69f4103450 2024-04-25T11:42:26.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.985+0000 7f69f3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f4102a90 0x7f69f4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.985+0000 7f69f3fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f4102a90 0x7f69f4074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60084/0 (socket says 172.21.15.83:60084) 2024-04-25T11:42:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.985+0000 7f69f3fff700 1 -- 172.21.15.83:0/3109814026 learned_addr learned my addr 172.21.15.83:0/3109814026 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.985+0000 7f69f37fe700 1 --2- 172.21.15.83:0/3109814026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69f4103450 0x7f69f4075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.986+0000 7f69f8fb8700 1 --2- 172.21.15.83:0/3109814026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f4108a50 0x7f69f40757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.986+0000 7f69f3fff700 1 -- 172.21.15.83:0/3109814026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69f4103450 msgr2=0x7f69f4075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.986+0000 7f69f3fff700 1 --2- 172.21.15.83:0/3109814026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69f4103450 0x7f69f4075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.986+0000 7f69f3fff700 1 -- 172.21.15.83:0/3109814026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f4108a50 msgr2=0x7f69f40757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.986+0000 7f69f3fff700 1 --2- 172.21.15.83:0/3109814026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f4108a50 0x7f69f40757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.986+0000 7f69f3fff700 1 -- 172.21.15.83:0/3109814026 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f41a76f0 con 0x7f69f4102a90 2024-04-25T11:42:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.986+0000 7f69f3fff700 1 --2- 172.21.15.83:0/3109814026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f4102a90 0x7f69f4074d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f69e400b530 tx=0x7f69e400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.986+0000 7f69f17fa700 1 -- 172.21.15.83:0/3109814026 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4015020 con 0x7f69f4102a90 2024-04-25T11:42:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.986+0000 7f69f17fa700 1 -- 172.21.15.83:0/3109814026 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69e4011ce0 con 0x7f69f4102a90 2024-04-25T11:42:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.986+0000 7f69faa1b700 1 -- 172.21.15.83:0/3109814026 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f41a79e0 con 0x7f69f4102a90 2024-04-25T11:42:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.986+0000 7f69f17fa700 1 -- 172.21.15.83:0/3109814026 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4010e40 con 0x7f69f4102a90 2024-04-25T11:42:26.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.986+0000 7f69faa1b700 1 -- 172.21.15.83:0/3109814026 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f41a7f20 con 0x7f69f4102a90 2024-04-25T11:42:26.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.987+0000 7f69faa1b700 1 -- 172.21.15.83:0/3109814026 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69f4063110 con 0x7f69f4102a90 2024-04-25T11:42:26.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.988+0000 7f69f17fa700 1 -- 172.21.15.83:0/3109814026 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f69e4018020 con 0x7f69f4102a90 2024-04-25T11:42:26.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.989+0000 7f69f17fa700 1 --2- 172.21.15.83:0/3109814026 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f69dc0a3cd0 0x7f69dc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.989+0000 7f69f37fe700 1 --2- 172.21.15.83:0/3109814026 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f69dc0a3cd0 0x7f69dc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.989+0000 7f69f17fa700 1 -- 172.21.15.83:0/3109814026 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f69e40cfaf0 con 0x7f69f4102a90 2024-04-25T11:42:26.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.989+0000 7f69f17fa700 1 --2- 172.21.15.83:0/3109814026 >> [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032] conn(0x7f69dc0ad4b0 0x7f69dc0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:26.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.989+0000 7f69f17fa700 1 -- 172.21.15.83:0/3109814026 --> [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69dc0aff80 con 0x7f69dc0ad4b0 2024-04-25T11:42:26.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.989+0000 7f69f17fa700 1 -- 172.21.15.83:0/3109814026 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f69e40d4260 con 0x7f69f4102a90 2024-04-25T11:42:26.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.989+0000 7f69f37fe700 1 --2- 172.21.15.83:0/3109814026 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f69dc0a3cd0 0x7f69dc0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f69f407a280 tx=0x7f69e800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:26.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.989+0000 7f69f8fb8700 1 --2- 172.21.15.83:0/3109814026 >> [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032] conn(0x7f69dc0ad4b0 0x7f69dc0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:26.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.990+0000 7f69f8fb8700 1 --2- 172.21.15.83:0/3109814026 >> [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032] conn(0x7f69dc0ad4b0 0x7f69dc0af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:26.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:26.991+0000 7f69f17fa700 1 -- 172.21.15.83:0/3109814026 <== osd.8 v2:172.21.15.111:6816/297826032 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f69dc0aff80 con 0x7f69dc0ad4b0 2024-04-25T11:42:26.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.007+0000 7f69faa1b700 1 -- 172.21.15.83:0/3109814026 --> [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f69f4063510 con 0x7f69dc0ad4b0 2024-04-25T11:42:26.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.007+0000 7f69f17fa700 1 -- 172.21.15.83:0/3109814026 <== osd.8 v2:172.21.15.111:6816/297826032 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f69f4063510 con 0x7f69dc0ad4b0 2024-04-25T11:42:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.008+0000 7f69faa1b700 1 -- 172.21.15.83:0/3109814026 >> [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032] conn(0x7f69dc0ad4b0 msgr2=0x7f69dc0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.008+0000 7f69faa1b700 1 --2- 172.21.15.83:0/3109814026 >> [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032] conn(0x7f69dc0ad4b0 0x7f69dc0af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.008+0000 7f69faa1b700 1 -- 172.21.15.83:0/3109814026 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f69dc0a3cd0 msgr2=0x7f69dc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.008+0000 7f69faa1b700 1 --2- 172.21.15.83:0/3109814026 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f69dc0a3cd0 0x7f69dc0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f69f407a280 tx=0x7f69e800a380 comp rx=0 tx=0).stop 2024-04-25T11:42:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.008+0000 7f69faa1b700 1 -- 172.21.15.83:0/3109814026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f4102a90 msgr2=0x7f69f4074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.008+0000 7f69faa1b700 1 --2- 172.21.15.83:0/3109814026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69f4102a90 0x7f69f4074d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f69e400b530 tx=0x7f69e400b840 comp rx=0 tx=0).stop 2024-04-25T11:42:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.008+0000 7f69f8fb8700 1 -- 172.21.15.83:0/3109814026 reap_dead start 2024-04-25T11:42:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.008+0000 7f69faa1b700 1 -- 172.21.15.83:0/3109814026 shutdown_connections 2024-04-25T11:42:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.008+0000 7f69faa1b700 1 -- 172.21.15.83:0/3109814026 >> 172.21.15.83:0/3109814026 conn(0x7f69f4079ba0 msgr2=0x7f69f4101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.008+0000 7f69faa1b700 1 -- 172.21.15.83:0/3109814026 shutdown_connections 2024-04-25T11:42:26.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.008+0000 7f69faa1b700 1 -- 172.21.15.83:0/3109814026 wait complete. 2024-04-25T11:42:27.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:27 smithi083 ceph-mon[29330]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T11:42:27.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:27 smithi094 ceph-mon[33032]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T11:42:27.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:27 smithi111 ceph-mon[32784]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T11:42:27.404 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.646+0000 7f17f43c4700 1 -- 172.21.15.83:0/2260681988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17ec108a50 msgr2=0x7f17ec108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.646+0000 7f17f43c4700 1 --2- 172.21.15.83:0/2260681988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17ec108a50 0x7f17ec108e30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009990 tx=0x7f17e8009ca0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.646+0000 7f17f43c4700 1 -- 172.21.15.83:0/2260681988 shutdown_connections 2024-04-25T11:42:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.646+0000 7f17f43c4700 1 --2- 172.21.15.83:0/2260681988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17ec103450 0x7f17ec10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.646+0000 7f17f43c4700 1 --2- 172.21.15.83:0/2260681988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17ec108a50 0x7f17ec108e30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.646+0000 7f17f43c4700 1 --2- 172.21.15.83:0/2260681988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17ec102a90 0x7f17ec102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.646+0000 7f17f43c4700 1 -- 172.21.15.83:0/2260681988 >> 172.21.15.83:0/2260681988 conn(0x7f17ec079ba0 msgr2=0x7f17ec079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:27.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.646+0000 7f17f43c4700 1 -- 172.21.15.83:0/2260681988 shutdown_connections 2024-04-25T11:42:27.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.647+0000 7f17f43c4700 1 -- 172.21.15.83:0/2260681988 wait complete. 2024-04-25T11:42:27.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.647+0000 7f17f43c4700 1 Processor -- start 2024-04-25T11:42:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.647+0000 7f17f43c4700 1 -- start start 2024-04-25T11:42:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f43c4700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17ec102a90 0x7f17ec074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f43c4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17ec103450 0x7f17ec075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f2160700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17ec102a90 0x7f17ec074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f43c4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17ec108a50 0x7f17ec0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f43c4700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec112be0 con 0x7f17ec102a90 2024-04-25T11:42:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f43c4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec112a60 con 0x7f17ec103450 2024-04-25T11:42:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f43c4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec112d60 con 0x7f17ec108a50 2024-04-25T11:42:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f2160700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17ec102a90 0x7f17ec074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60098/0 (socket says 172.21.15.83:60098) 2024-04-25T11:42:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f2160700 1 -- 172.21.15.83:0/788545310 learned_addr learned my addr 172.21.15.83:0/788545310 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f195f700 1 --2- 172.21.15.83:0/788545310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17ec103450 0x7f17ec075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f2961700 1 --2- 172.21.15.83:0/788545310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17ec108a50 0x7f17ec0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f2160700 1 -- 172.21.15.83:0/788545310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17ec108a50 msgr2=0x7f17ec0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f2160700 1 --2- 172.21.15.83:0/788545310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17ec108a50 0x7f17ec0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f2160700 1 -- 172.21.15.83:0/788545310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17ec103450 msgr2=0x7f17ec075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f2160700 1 --2- 172.21.15.83:0/788545310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17ec103450 0x7f17ec075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f2160700 1 -- 172.21.15.83:0/788545310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ec1a76a0 con 0x7f17ec102a90 2024-04-25T11:42:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f195f700 1 --2- 172.21.15.83:0/788545310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f17ec103450 0x7f17ec075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:42:27.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17f2160700 1 --2- 172.21.15.83:0/788545310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17ec102a90 0x7f17ec074d40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f17e8000c00 tx=0x7f17e80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.648+0000 7f17e37fe700 1 -- 172.21.15.83:0/788545310 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8027070 con 0x7f17ec102a90 2024-04-25T11:42:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.649+0000 7f17f43c4700 1 -- 172.21.15.83:0/788545310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17ec1a7930 con 0x7f17ec102a90 2024-04-25T11:42:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.649+0000 7f17e37fe700 1 -- 172.21.15.83:0/788545310 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e8003fa0 con 0x7f17ec102a90 2024-04-25T11:42:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.649+0000 7f17e37fe700 1 -- 172.21.15.83:0/788545310 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8022db0 con 0x7f17ec102a90 2024-04-25T11:42:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.649+0000 7f17f43c4700 1 -- 172.21.15.83:0/788545310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17ec1a7e10 con 0x7f17ec102a90 2024-04-25T11:42:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.649+0000 7f17f43c4700 1 -- 172.21.15.83:0/788545310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f17ec063110 con 0x7f17ec102a90 2024-04-25T11:42:27.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.651+0000 7f17e37fe700 1 -- 172.21.15.83:0/788545310 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f17e802a070 con 0x7f17ec102a90 2024-04-25T11:42:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.651+0000 7f17e37fe700 1 --2- 172.21.15.83:0/788545310 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f17d80a3cd0 0x7f17d80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.651+0000 7f17f195f700 1 --2- 172.21.15.83:0/788545310 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f17d80a3cd0 0x7f17d80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.651+0000 7f17e37fe700 1 -- 172.21.15.83:0/788545310 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f17e80d1710 con 0x7f17ec102a90 2024-04-25T11:42:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.651+0000 7f17e37fe700 1 --2- 172.21.15.83:0/788545310 >> [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764] conn(0x7f17d80ad4b0 0x7f17d80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.651+0000 7f17e37fe700 1 -- 172.21.15.83:0/788545310 --> [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17e8016e70 con 0x7f17d80ad4b0 2024-04-25T11:42:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.651+0000 7f17e37fe700 1 -- 172.21.15.83:0/788545310 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f17e80d1bf0 con 0x7f17ec102a90 2024-04-25T11:42:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.652+0000 7f17f2961700 1 --2- 172.21.15.83:0/788545310 >> [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764] conn(0x7f17d80ad4b0 0x7f17d80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.652+0000 7f17f195f700 1 --2- 172.21.15.83:0/788545310 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f17d80a3cd0 0x7f17d80a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f17dc005950 tx=0x7f17dc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.652+0000 7f17f2961700 1 --2- 172.21.15.83:0/788545310 >> [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764] conn(0x7f17d80ad4b0 0x7f17d80af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:27.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.653+0000 7f17e37fe700 1 -- 172.21.15.83:0/788545310 <== osd.2 v2:172.21.15.111:6800/101429764 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17e8016e70 con 0x7f17d80ad4b0 2024-04-25T11:42:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.669+0000 7f17f43c4700 1 -- 172.21.15.83:0/788545310 --> [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f17ec063350 con 0x7f17d80ad4b0 2024-04-25T11:42:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.669+0000 7f17e37fe700 1 -- 172.21.15.83:0/788545310 <== osd.2 v2:172.21.15.111:6800/101429764 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f17ec063350 con 0x7f17d80ad4b0 2024-04-25T11:42:27.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.670+0000 7f17f43c4700 1 -- 172.21.15.83:0/788545310 >> [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764] conn(0x7f17d80ad4b0 msgr2=0x7f17d80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:27.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.670+0000 7f17f43c4700 1 --2- 172.21.15.83:0/788545310 >> [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764] conn(0x7f17d80ad4b0 0x7f17d80af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.670+0000 7f17f43c4700 1 -- 172.21.15.83:0/788545310 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f17d80a3cd0 msgr2=0x7f17d80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:27.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.670+0000 7f17f43c4700 1 --2- 172.21.15.83:0/788545310 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f17d80a3cd0 0x7f17d80a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f17dc005950 tx=0x7f17dc0058e0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.670+0000 7f17f43c4700 1 -- 172.21.15.83:0/788545310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17ec102a90 msgr2=0x7f17ec074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:27.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.670+0000 7f17f43c4700 1 --2- 172.21.15.83:0/788545310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17ec102a90 0x7f17ec074d40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f17e8000c00 tx=0x7f17e80047b0 comp rx=0 tx=0).stop 2024-04-25T11:42:27.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.670+0000 7f17f2961700 1 -- 172.21.15.83:0/788545310 reap_dead start 2024-04-25T11:42:27.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.670+0000 7f17f43c4700 1 -- 172.21.15.83:0/788545310 shutdown_connections 2024-04-25T11:42:27.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.670+0000 7f17f43c4700 1 -- 172.21.15.83:0/788545310 >> 172.21.15.83:0/788545310 conn(0x7f17ec079ba0 msgr2=0x7f17ec1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:27.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.670+0000 7f17f43c4700 1 -- 172.21.15.83:0/788545310 shutdown_connections 2024-04-25T11:42:27.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:27.670+0000 7f17f43c4700 1 -- 172.21.15.83:0/788545310 wait complete. 2024-04-25T11:42:28.493 INFO:teuthology.orchestra.run.smithi083.stdout:223338299433 2024-04-25T11:42:28.494 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T11:42:28.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.914+0000 7fd5e02e0700 1 -- 172.21.15.83:0/2013022176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5d8101870 msgr2=0x7fd5d81058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.914+0000 7fd5e02e0700 1 --2- 172.21.15.83:0/2013022176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5d8101870 0x7fd5d81058c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80099c0 tx=0x7fd5c8009cd0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.914+0000 7fd5e02e0700 1 -- 172.21.15.83:0/2013022176 shutdown_connections 2024-04-25T11:42:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.914+0000 7fd5e02e0700 1 --2- 172.21.15.83:0/2013022176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d8105fa0 0x7fd5d810db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.914+0000 7fd5e02e0700 1 --2- 172.21.15.83:0/2013022176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5d8100ec0 0x7fd5d81012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.914+0000 7fd5e02e0700 1 --2- 172.21.15.83:0/2013022176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5d8101870 0x7fd5d81058c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.914+0000 7fd5e02e0700 1 -- 172.21.15.83:0/2013022176 >> 172.21.15.83:0/2013022176 conn(0x7fd5d8077b40 msgr2=0x7fd5d80feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.915+0000 7fd5e02e0700 1 -- 172.21.15.83:0/2013022176 shutdown_connections 2024-04-25T11:42:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.915+0000 7fd5e02e0700 1 -- 172.21.15.83:0/2013022176 wait complete. 2024-04-25T11:42:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.915+0000 7fd5e02e0700 1 Processor -- start 2024-04-25T11:42:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.915+0000 7fd5e02e0700 1 -- start start 2024-04-25T11:42:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.915+0000 7fd5e02e0700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5d8100ec0 0x7fd5d8199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.915+0000 7fd5e02e0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d8101870 0x7fd5d8199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:28.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.915+0000 7fd5e02e0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5d8105fa0 0x7fd5d819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.915+0000 7fd5e02e0700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d8111350 con 0x7fd5d8100ec0 2024-04-25T11:42:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.915+0000 7fd5e02e0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d81111d0 con 0x7fd5d8101870 2024-04-25T11:42:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.915+0000 7fd5e02e0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d81114d0 con 0x7fd5d8105fa0 2024-04-25T11:42:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.915+0000 7fd5de07c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5d8100ec0 0x7fd5d8199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.915+0000 7fd5de07c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5d8100ec0 0x7fd5d8199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60114/0 (socket says 172.21.15.83:60114) 2024-04-25T11:42:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.915+0000 7fd5de07c700 1 -- 172.21.15.83:0/3924704260 learned_addr learned my addr 172.21.15.83:0/3924704260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.916+0000 7fd5dd87b700 1 --2- 172.21.15.83:0/3924704260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d8101870 0x7fd5d8199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.916+0000 7fd5de07c700 1 -- 172.21.15.83:0/3924704260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5d8105fa0 msgr2=0x7fd5d819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.916+0000 7fd5de07c700 1 --2- 172.21.15.83:0/3924704260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5d8105fa0 0x7fd5d819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.916+0000 7fd5de07c700 1 -- 172.21.15.83:0/3924704260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d8101870 msgr2=0x7fd5d8199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.916+0000 7fd5de07c700 1 --2- 172.21.15.83:0/3924704260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5d8101870 0x7fd5d8199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.916+0000 7fd5de07c700 1 -- 172.21.15.83:0/3924704260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d819e210 con 0x7fd5d8100ec0 2024-04-25T11:42:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.916+0000 7fd5de07c700 1 --2- 172.21.15.83:0/3924704260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5d8100ec0 0x7fd5d8199200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400b530 tx=0x7fd5d400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:28.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.916+0000 7fd5cf7fe700 1 -- 172.21.15.83:0/3924704260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4015020 con 0x7fd5d8100ec0 2024-04-25T11:42:28.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.916+0000 7fd5cf7fe700 1 -- 172.21.15.83:0/3924704260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d4011ce0 con 0x7fd5d8100ec0 2024-04-25T11:42:28.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.916+0000 7fd5e02e0700 1 -- 172.21.15.83:0/3924704260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d819e500 con 0x7fd5d8100ec0 2024-04-25T11:42:28.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.916+0000 7fd5cf7fe700 1 -- 172.21.15.83:0/3924704260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4010d20 con 0x7fd5d8100ec0 2024-04-25T11:42:28.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.916+0000 7fd5e02e0700 1 -- 172.21.15.83:0/3924704260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d81a5d80 con 0x7fd5d8100ec0 2024-04-25T11:42:28.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.917+0000 7fd5e02e0700 1 -- 172.21.15.83:0/3924704260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd5d81124f0 con 0x7fd5d8100ec0 2024-04-25T11:42:28.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.918+0000 7fd5cf7fe700 1 -- 172.21.15.83:0/3924704260 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd5d4018020 con 0x7fd5d8100ec0 2024-04-25T11:42:28.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.919+0000 7fd5cf7fe700 1 --2- 172.21.15.83:0/3924704260 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd5c40a3d50 0x7fd5c40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:28.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.919+0000 7fd5dd87b700 1 --2- 172.21.15.83:0/3924704260 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd5c40a3d50 0x7fd5c40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:28.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.919+0000 7fd5cf7fe700 1 -- 172.21.15.83:0/3924704260 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd5d40d0160 con 0x7fd5d8100ec0 2024-04-25T11:42:28.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.919+0000 7fd5cf7fe700 1 --2- 172.21.15.83:0/3924704260 >> [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468] conn(0x7fd5c40ad530 0x7fd5c40af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:28.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.919+0000 7fd5cf7fe700 1 -- 172.21.15.83:0/3924704260 --> [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5d401deb0 con 0x7fd5c40ad530 2024-04-25T11:42:28.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.919+0000 7fd5cf7fe700 1 -- 172.21.15.83:0/3924704260 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fd5d40d0640 con 0x7fd5d8100ec0 2024-04-25T11:42:28.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.919+0000 7fd5dd87b700 1 --2- 172.21.15.83:0/3924704260 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd5c40a3d50 0x7fd5c40a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8006010 tx=0x7fd5c8009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:28.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.919+0000 7fd5de87d700 1 --2- 172.21.15.83:0/3924704260 >> [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468] conn(0x7fd5c40ad530 0x7fd5c40af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:28.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.920+0000 7fd5de87d700 1 --2- 172.21.15.83:0/3924704260 >> [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468] conn(0x7fd5c40ad530 0x7fd5c40af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:28.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.921+0000 7fd5cf7fe700 1 -- 172.21.15.83:0/3924704260 <== osd.11 v2:172.21.15.111:6824/4117294468 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd5d401deb0 con 0x7fd5c40ad530 2024-04-25T11:42:28.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.938+0000 7fd5e02e0700 1 -- 172.21.15.83:0/3924704260 --> [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd5d81023a0 con 0x7fd5c40ad530 2024-04-25T11:42:28.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.938+0000 7fd5cf7fe700 1 -- 172.21.15.83:0/3924704260 <== osd.11 v2:172.21.15.111:6824/4117294468 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd5d81023a0 con 0x7fd5c40ad530 2024-04-25T11:42:28.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.938+0000 7fd5e02e0700 1 -- 172.21.15.83:0/3924704260 >> [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468] conn(0x7fd5c40ad530 msgr2=0x7fd5c40af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:28.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.938+0000 7fd5e02e0700 1 --2- 172.21.15.83:0/3924704260 >> [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468] conn(0x7fd5c40ad530 0x7fd5c40af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.939+0000 7fd5e02e0700 1 -- 172.21.15.83:0/3924704260 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd5c40a3d50 msgr2=0x7fd5c40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:28.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.939+0000 7fd5e02e0700 1 --2- 172.21.15.83:0/3924704260 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd5c40a3d50 0x7fd5c40a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8006010 tx=0x7fd5c8009f10 comp rx=0 tx=0).stop 2024-04-25T11:42:28.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.939+0000 7fd5e02e0700 1 -- 172.21.15.83:0/3924704260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5d8100ec0 msgr2=0x7fd5d8199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:28.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.939+0000 7fd5e02e0700 1 --2- 172.21.15.83:0/3924704260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd5d8100ec0 0x7fd5d8199200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400b530 tx=0x7fd5d400b8f0 comp rx=0 tx=0).stop 2024-04-25T11:42:28.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.939+0000 7fd5de87d700 1 -- 172.21.15.83:0/3924704260 reap_dead start 2024-04-25T11:42:28.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.939+0000 7fd5e02e0700 1 -- 172.21.15.83:0/3924704260 shutdown_connections 2024-04-25T11:42:28.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.939+0000 7fd5e02e0700 1 -- 172.21.15.83:0/3924704260 >> 172.21.15.83:0/3924704260 conn(0x7fd5d8077b40 msgr2=0x7fd5d8111b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:28.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.939+0000 7fd5e02e0700 1 -- 172.21.15.83:0/3924704260 shutdown_connections 2024-04-25T11:42:28.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:28.939+0000 7fd5e02e0700 1 -- 172.21.15.83:0/3924704260 wait complete. 2024-04-25T11:42:29.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:29 smithi083 ceph-mon[29330]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:42:29.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:29 smithi094 ceph-mon[33032]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:42:29.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:29 smithi111 ceph-mon[32784]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:42:29.407 INFO:teuthology.orchestra.run.smithi083.stdout:77309411393 2024-04-25T11:42:29.408 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T11:42:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:29.472 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:29.682 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:30.085 INFO:teuthology.orchestra.run.smithi083.stdout:347892350996 2024-04-25T11:42:30.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T11:42:30.578 INFO:teuthology.orchestra.run.smithi083.stdout:257698037795 2024-04-25T11:42:30.579 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T11:42:31.074 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:31 smithi083 ceph-mon[29330]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T11:42:31.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:31 smithi094 ceph-mon[33032]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T11:42:31.273 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:31.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:31 smithi111 ceph-mon[32784]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T11:42:33.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:33 smithi094 ceph-mon[33032]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:42:33.305 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:33 smithi083 ceph-mon[29330]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:42:33.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:33 smithi111 ceph-mon[32784]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:42:35.096 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:35 smithi083 ceph-mon[29330]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T11:42:35.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:35 smithi094 ceph-mon[33032]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T11:42:35.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:35 smithi111 ceph-mon[32784]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T11:42:37.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:37 smithi094 ceph-mon[33032]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T11:42:37.315 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:37 smithi083 ceph-mon[29330]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T11:42:37.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:37 smithi111 ceph-mon[32784]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T11:42:37.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.782+0000 7f4dae85e700 1 -- 172.21.15.83:0/2774771623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4da8108a50 msgr2=0x7f4da8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:37.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.782+0000 7f4dae85e700 1 --2- 172.21.15.83:0/2774771623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4da8108a50 0x7f4da8108e30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009990 tx=0x7f4d98009ca0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.782+0000 7f4dae85e700 1 -- 172.21.15.83:0/2774771623 shutdown_connections 2024-04-25T11:42:37.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.782+0000 7f4dae85e700 1 --2- 172.21.15.83:0/2774771623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da8103450 0x7f4da810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.782+0000 7f4dae85e700 1 --2- 172.21.15.83:0/2774771623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4da8108a50 0x7f4da8108e30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.782+0000 7f4dae85e700 1 --2- 172.21.15.83:0/2774771623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4da8102a90 0x7f4da8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.782+0000 7f4dae85e700 1 -- 172.21.15.83:0/2774771623 >> 172.21.15.83:0/2774771623 conn(0x7f4da8079ba0 msgr2=0x7f4da8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:37.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.782+0000 7f4dae85e700 1 -- 172.21.15.83:0/2774771623 shutdown_connections 2024-04-25T11:42:37.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.782+0000 7f4dae85e700 1 -- 172.21.15.83:0/2774771623 wait complete. 2024-04-25T11:42:37.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.782+0000 7f4dae85e700 1 Processor -- start 2024-04-25T11:42:37.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.782+0000 7f4dae85e700 1 -- start start 2024-04-25T11:42:37.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4dae85e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4da8102a90 0x7f4da8199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4dae85e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da8103450 0x7f4da81996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4dae85e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4da8108a50 0x7f4da819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4dae85e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4da8112c60 con 0x7f4da8102a90 2024-04-25T11:42:37.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4dae85e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4da8112ae0 con 0x7f4da8103450 2024-04-25T11:42:37.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4dae85e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4da8112de0 con 0x7f4da8108a50 2024-04-25T11:42:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4da7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4da8102a90 0x7f4da8199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4da7fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4da8102a90 0x7f4da8199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51242/0 (socket says 172.21.15.83:51242) 2024-04-25T11:42:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4da7fff700 1 -- 172.21.15.83:0/1462260996 learned_addr learned my addr 172.21.15.83:0/1462260996 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4da77fe700 1 --2- 172.21.15.83:0/1462260996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da8103450 0x7f4da81996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4da7fff700 1 -- 172.21.15.83:0/1462260996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4da8108a50 msgr2=0x7f4da819da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4da7fff700 1 --2- 172.21.15.83:0/1462260996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4da8108a50 0x7f4da819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4da7fff700 1 -- 172.21.15.83:0/1462260996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da8103450 msgr2=0x7f4da81996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4da7fff700 1 --2- 172.21.15.83:0/1462260996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da8103450 0x7f4da81996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4da7fff700 1 -- 172.21.15.83:0/1462260996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da819e1a0 con 0x7f4da8102a90 2024-04-25T11:42:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4da77fe700 1 --2- 172.21.15.83:0/1462260996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da8103450 0x7f4da81996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:42:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.783+0000 7f4da7fff700 1 --2- 172.21.15.83:0/1462260996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4da8102a90 0x7f4da8199190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4d98000c00 tx=0x7f4d980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.784+0000 7f4da57fa700 1 -- 172.21.15.83:0/1462260996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98027070 con 0x7f4da8102a90 2024-04-25T11:42:37.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.784+0000 7f4dae85e700 1 -- 172.21.15.83:0/1462260996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da819e430 con 0x7f4da8102a90 2024-04-25T11:42:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.784+0000 7f4da57fa700 1 -- 172.21.15.83:0/1462260996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d980041d0 con 0x7f4da8102a90 2024-04-25T11:42:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.784+0000 7f4da57fa700 1 -- 172.21.15.83:0/1462260996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800f460 con 0x7f4da8102a90 2024-04-25T11:42:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.784+0000 7f4dae85e700 1 -- 172.21.15.83:0/1462260996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da81a5d10 con 0x7f4da8102a90 2024-04-25T11:42:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.785+0000 7f4dae85e700 1 -- 172.21.15.83:0/1462260996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da8069730 con 0x7f4da8102a90 2024-04-25T11:42:37.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.786+0000 7f4da57fa700 1 -- 172.21.15.83:0/1462260996 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4d9802a070 con 0x7f4da8102a90 2024-04-25T11:42:37.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.786+0000 7f4da57fa700 1 --2- 172.21.15.83:0/1462260996 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4d900a3d50 0x7f4d900a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:37.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.786+0000 7f4da77fe700 1 --2- 172.21.15.83:0/1462260996 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4d900a3d50 0x7f4d900a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:37.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.787+0000 7f4da57fa700 1 -- 172.21.15.83:0/1462260996 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4d980d1770 con 0x7f4da8102a90 2024-04-25T11:42:37.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.787+0000 7f4da77fe700 1 --2- 172.21.15.83:0/1462260996 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4d900a3d50 0x7f4d900a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4da807a280 tx=0x7f4d9c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:37.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.790+0000 7f4da57fa700 1 -- 172.21.15.83:0/1462260996 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d98031070 con 0x7f4da8102a90 2024-04-25T11:42:37.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.997+0000 7f4dae85e700 1 -- 172.21.15.83:0/1462260996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f4da802df80 con 0x7f4da8102a90 2024-04-25T11:42:37.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:37.997+0000 7f4da57fa700 1 -- 172.21.15.83:0/1462260996 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f4d98013070 con 0x7f4da8102a90 2024-04-25T11:42:37.984 INFO:teuthology.orchestra.run.smithi083.stdout:313532612636 2024-04-25T11:42:37.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.000+0000 7f4dae85e700 1 -- 172.21.15.83:0/1462260996 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4d900a3d50 msgr2=0x7f4d900a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:37.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.000+0000 7f4dae85e700 1 --2- 172.21.15.83:0/1462260996 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4d900a3d50 0x7f4d900a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4da807a280 tx=0x7f4d9c008040 comp rx=0 tx=0).stop 2024-04-25T11:42:37.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.000+0000 7f4dae85e700 1 -- 172.21.15.83:0/1462260996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4da8102a90 msgr2=0x7f4da8199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.000+0000 7f4dae85e700 1 --2- 172.21.15.83:0/1462260996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4da8102a90 0x7f4da8199190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4d98000c00 tx=0x7f4d980047b0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.000+0000 7f4dae85e700 1 -- 172.21.15.83:0/1462260996 shutdown_connections 2024-04-25T11:42:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.000+0000 7f4dae85e700 1 --2- 172.21.15.83:0/1462260996 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4d900a3d50 0x7f4d900a6210 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.000+0000 7f4dae85e700 1 --2- 172.21.15.83:0/1462260996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4da8108a50 0x7f4da819da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.000+0000 7f4dae85e700 1 --2- 172.21.15.83:0/1462260996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4da8102a90 0x7f4da8199190 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.000+0000 7f4dae85e700 1 --2- 172.21.15.83:0/1462260996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4da8103450 0x7f4da81996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.000+0000 7f4dae85e700 1 -- 172.21.15.83:0/1462260996 >> 172.21.15.83:0/1462260996 conn(0x7f4da8079ba0 msgr2=0x7f4da8101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.000+0000 7f4dae85e700 1 -- 172.21.15.83:0/1462260996 shutdown_connections 2024-04-25T11:42:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.000+0000 7f4dae85e700 1 -- 172.21.15.83:0/1462260996 wait complete. 2024-04-25T11:42:38.078 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:38 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1462260996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T11:42:38.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:38 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1462260996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T11:42:38.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:38 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1462260996' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T11:42:38.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.720+0000 7fa31184a700 1 -- 172.21.15.83:0/3134074869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30c102300 msgr2=0x7fa30c10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.720+0000 7fa31184a700 1 --2- 172.21.15.83:0/3134074869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30c102300 0x7fa30c10e7f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa30400b0a0 tx=0x7fa30400b3b0 comp rx=0 tx=0).stop 2024-04-25T11:42:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.720+0000 7fa31184a700 1 -- 172.21.15.83:0/3134074869 shutdown_connections 2024-04-25T11:42:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.720+0000 7fa31184a700 1 --2- 172.21.15.83:0/3134074869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30c102300 0x7fa30c10e7f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.720+0000 7fa31184a700 1 --2- 172.21.15.83:0/3134074869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa30c107990 0x7fa30c107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.720+0000 7fa31184a700 1 --2- 172.21.15.83:0/3134074869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa30c101940 0x7fa30c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.720+0000 7fa31184a700 1 -- 172.21.15.83:0/3134074869 >> 172.21.15.83:0/3134074869 conn(0x7fa30c0788f0 msgr2=0x7fa30c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.720+0000 7fa31184a700 1 -- 172.21.15.83:0/3134074869 shutdown_connections 2024-04-25T11:42:38.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.720+0000 7fa31184a700 1 -- 172.21.15.83:0/3134074869 wait complete. 2024-04-25T11:42:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.721+0000 7fa31184a700 1 Processor -- start 2024-04-25T11:42:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.721+0000 7fa31184a700 1 -- start start 2024-04-25T11:42:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.721+0000 7fa31184a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30c101940 0x7fa30c199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.721+0000 7fa31184a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa30c102300 0x7fa30c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.721+0000 7fa31184a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa30c107990 0x7fa30c1a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.721+0000 7fa31184a700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c111b60 con 0x7fa30c101940 2024-04-25T11:42:38.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.721+0000 7fa31184a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c1119e0 con 0x7fa30c102300 2024-04-25T11:42:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.721+0000 7fa31184a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c111ce0 con 0x7fa30c107990 2024-04-25T11:42:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.721+0000 7fa30affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30c101940 0x7fa30c199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.721+0000 7fa30affd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30c101940 0x7fa30c199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51262/0 (socket says 172.21.15.83:51262) 2024-04-25T11:42:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.721+0000 7fa30affd700 1 -- 172.21.15.83:0/433221409 learned_addr learned my addr 172.21.15.83:0/433221409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.721+0000 7fa30b7fe700 1 --2- 172.21.15.83:0/433221409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa30c107990 0x7fa30c1a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.721+0000 7fa30a7fc700 1 --2- 172.21.15.83:0/433221409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa30c102300 0x7fa30c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.722+0000 7fa30affd700 1 -- 172.21.15.83:0/433221409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa30c107990 msgr2=0x7fa30c1a4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.722+0000 7fa30affd700 1 --2- 172.21.15.83:0/433221409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa30c107990 0x7fa30c1a4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.722+0000 7fa30affd700 1 -- 172.21.15.83:0/433221409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa30c102300 msgr2=0x7fa30c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.722+0000 7fa30affd700 1 --2- 172.21.15.83:0/433221409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa30c102300 0x7fa30c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.722+0000 7fa30affd700 1 -- 172.21.15.83:0/433221409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c1a54a0 con 0x7fa30c101940 2024-04-25T11:42:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.722+0000 7fa30a7fc700 1 --2- 172.21.15.83:0/433221409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa30c102300 0x7fa30c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:42:38.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.722+0000 7fa30affd700 1 --2- 172.21.15.83:0/433221409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30c101940 0x7fa30c199ee0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00b530 tx=0x7fa2fc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:38.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.722+0000 7fa310848700 1 -- 172.21.15.83:0/433221409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc015020 con 0x7fa30c101940 2024-04-25T11:42:38.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.722+0000 7fa310848700 1 -- 172.21.15.83:0/433221409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2fc011ce0 con 0x7fa30c101940 2024-04-25T11:42:38.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.722+0000 7fa31184a700 1 -- 172.21.15.83:0/433221409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30c1a56d0 con 0x7fa30c101940 2024-04-25T11:42:38.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.722+0000 7fa310848700 1 -- 172.21.15.83:0/433221409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc010e40 con 0x7fa30c101940 2024-04-25T11:42:38.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.722+0000 7fa31184a700 1 -- 172.21.15.83:0/433221409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30c1a5bc0 con 0x7fa30c101940 2024-04-25T11:42:38.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.723+0000 7fa31184a700 1 -- 172.21.15.83:0/433221409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa30c109eb0 con 0x7fa30c101940 2024-04-25T11:42:38.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.724+0000 7fa310848700 1 -- 172.21.15.83:0/433221409 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa2fc018020 con 0x7fa30c101940 2024-04-25T11:42:38.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.724+0000 7fa310848700 1 --2- 172.21.15.83:0/433221409 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa2f80a3cd0 0x7fa2f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:38.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.725+0000 7fa30a7fc700 1 --2- 172.21.15.83:0/433221409 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa2f80a3cd0 0x7fa2f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:38.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.725+0000 7fa310848700 1 -- 172.21.15.83:0/433221409 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa2fc0cfb40 con 0x7fa30c101940 2024-04-25T11:42:38.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.725+0000 7fa30a7fc700 1 --2- 172.21.15.83:0/433221409 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa2f80a3cd0 0x7fa2f80a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4005fd0 tx=0x7fa2f4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:38.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.729+0000 7fa310848700 1 -- 172.21.15.83:0/433221409 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2fc01f020 con 0x7fa30c101940 2024-04-25T11:42:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.932+0000 7fa31184a700 1 -- 172.21.15.83:0/433221409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa30c109890 con 0x7fa30c101940 2024-04-25T11:42:38.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.932+0000 7fa310848700 1 -- 172.21.15.83:0/433221409 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa2fc09c680 con 0x7fa30c101940 2024-04-25T11:42:38.919 INFO:teuthology.orchestra.run.smithi083.stdout:111669149759 2024-04-25T11:42:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.934+0000 7fa31184a700 1 -- 172.21.15.83:0/433221409 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa2f80a3cd0 msgr2=0x7fa2f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.934+0000 7fa31184a700 1 --2- 172.21.15.83:0/433221409 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa2f80a3cd0 0x7fa2f80a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4005fd0 tx=0x7fa2f4005e30 comp rx=0 tx=0).stop 2024-04-25T11:42:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.934+0000 7fa31184a700 1 -- 172.21.15.83:0/433221409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30c101940 msgr2=0x7fa30c199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.934+0000 7fa31184a700 1 --2- 172.21.15.83:0/433221409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30c101940 0x7fa30c199ee0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00b530 tx=0x7fa2fc00b840 comp rx=0 tx=0).stop 2024-04-25T11:42:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.934+0000 7fa31184a700 1 -- 172.21.15.83:0/433221409 shutdown_connections 2024-04-25T11:42:38.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.934+0000 7fa31184a700 1 --2- 172.21.15.83:0/433221409 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa2f80a3cd0 0x7fa2f80a6190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:38.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.934+0000 7fa31184a700 1 --2- 172.21.15.83:0/433221409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa30c107990 0x7fa30c1a4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:38.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.934+0000 7fa31184a700 1 --2- 172.21.15.83:0/433221409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30c101940 0x7fa30c199ee0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:38.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.934+0000 7fa31184a700 1 --2- 172.21.15.83:0/433221409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa30c102300 0x7fa30c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:38.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.935+0000 7fa31184a700 1 -- 172.21.15.83:0/433221409 >> 172.21.15.83:0/433221409 conn(0x7fa30c0788f0 msgr2=0x7fa30c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:38.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.935+0000 7fa31184a700 1 -- 172.21.15.83:0/433221409 shutdown_connections 2024-04-25T11:42:38.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:38.935+0000 7fa31184a700 1 -- 172.21.15.83:0/433221409 wait complete. 2024-04-25T11:42:39.201 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:39 smithi083 ceph-mon[29330]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:42:39.202 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:39 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/433221409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T11:42:39.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:39 smithi111 ceph-mon[32784]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:42:39.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:39 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/433221409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T11:42:39.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.451+0000 7fc38d133700 1 -- 172.21.15.83:0/4120569239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388101940 msgr2=0x7fc388101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:39.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.451+0000 7fc38d133700 1 --2- 172.21.15.83:0/4120569239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388101940 0x7fc388101dc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc378009990 tx=0x7fc378009ca0 comp rx=0 tx=0).stop 2024-04-25T11:42:39.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.451+0000 7fc38d133700 1 -- 172.21.15.83:0/4120569239 shutdown_connections 2024-04-25T11:42:39.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.451+0000 7fc38d133700 1 --2- 172.21.15.83:0/4120569239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc388102300 0x7fc38810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:39.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.451+0000 7fc38d133700 1 --2- 172.21.15.83:0/4120569239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc388107990 0x7fc388107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:39.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.451+0000 7fc38d133700 1 --2- 172.21.15.83:0/4120569239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388101940 0x7fc388101dc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:39.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.451+0000 7fc38d133700 1 -- 172.21.15.83:0/4120569239 >> 172.21.15.83:0/4120569239 conn(0x7fc3880788f0 msgr2=0x7fc388078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:39.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.451+0000 7fc38d133700 1 -- 172.21.15.83:0/4120569239 shutdown_connections 2024-04-25T11:42:39.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.451+0000 7fc38d133700 1 -- 172.21.15.83:0/4120569239 wait complete. 2024-04-25T11:42:39.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc38d133700 1 Processor -- start 2024-04-25T11:42:39.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc38d133700 1 -- start start 2024-04-25T11:42:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc38d133700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388101940 0x7fc3881991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc38d133700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc388102300 0x7fc388199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc38d133700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc388107990 0x7fc38819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc38d133700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc388111b60 con 0x7fc388101940 2024-04-25T11:42:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc38d133700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc3881119e0 con 0x7fc388102300 2024-04-25T11:42:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc38d133700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc388111ce0 con 0x7fc388107990 2024-04-25T11:42:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc386d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388101940 0x7fc3881991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc386d9d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388101940 0x7fc3881991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51272/0 (socket says 172.21.15.83:51272) 2024-04-25T11:42:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc386d9d700 1 -- 172.21.15.83:0/2946031295 learned_addr learned my addr 172.21.15.83:0/2946031295 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc38659c700 1 --2- 172.21.15.83:0/2946031295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc388102300 0x7fc388199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc386d9d700 1 -- 172.21.15.83:0/2946031295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc388107990 msgr2=0x7fc38819da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc386d9d700 1 --2- 172.21.15.83:0/2946031295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc388107990 0x7fc38819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:39.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc386d9d700 1 -- 172.21.15.83:0/2946031295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc388102300 msgr2=0x7fc388199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:39.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc386d9d700 1 --2- 172.21.15.83:0/2946031295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc388102300 0x7fc388199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:39.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.452+0000 7fc386d9d700 1 -- 172.21.15.83:0/2946031295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38819e1a0 con 0x7fc388101940 2024-04-25T11:42:39.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.453+0000 7fc386d9d700 1 --2- 172.21.15.83:0/2946031295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388101940 0x7fc3881991e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00b530 tx=0x7fc37c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:39.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.453+0000 7fc36ffff700 1 -- 172.21.15.83:0/2946031295 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c015020 con 0x7fc388101940 2024-04-25T11:42:39.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.453+0000 7fc38d133700 1 -- 172.21.15.83:0/2946031295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc38819e490 con 0x7fc388101940 2024-04-25T11:42:39.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.453+0000 7fc36ffff700 1 -- 172.21.15.83:0/2946031295 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc37c011ce0 con 0x7fc388101940 2024-04-25T11:42:39.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.453+0000 7fc36ffff700 1 -- 172.21.15.83:0/2946031295 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c010e40 con 0x7fc388101940 2024-04-25T11:42:39.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.453+0000 7fc38d133700 1 -- 172.21.15.83:0/2946031295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3881a5d10 con 0x7fc388101940 2024-04-25T11:42:39.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.454+0000 7fc38d133700 1 -- 172.21.15.83:0/2946031295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc388069730 con 0x7fc388101940 2024-04-25T11:42:39.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.455+0000 7fc36ffff700 1 -- 172.21.15.83:0/2946031295 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc37c018020 con 0x7fc388101940 2024-04-25T11:42:39.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.455+0000 7fc36ffff700 1 --2- 172.21.15.83:0/2946031295 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc3700a3cd0 0x7fc3700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:39.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.456+0000 7fc38659c700 1 --2- 172.21.15.83:0/2946031295 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc3700a3cd0 0x7fc3700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:39.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.456+0000 7fc36ffff700 1 -- 172.21.15.83:0/2946031295 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc37c0d0080 con 0x7fc388101940 2024-04-25T11:42:39.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.456+0000 7fc38659c700 1 --2- 172.21.15.83:0/2946031295 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc3700a3cd0 0x7fc3700a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc37800b5c0 tx=0x7fc378005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:39.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.459+0000 7fc36ffff700 1 -- 172.21.15.83:0/2946031295 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc37c01f020 con 0x7fc388101940 2024-04-25T11:42:39.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:39 smithi094 ceph-mon[33032]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:42:39.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:39 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/433221409' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T11:42:39.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.669+0000 7fc38d133700 1 -- 172.21.15.83:0/2946031295 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc38819a4f0 con 0x7fc388101940 2024-04-25T11:42:39.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.669+0000 7fc36ffff700 1 -- 172.21.15.83:0/2946031295 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc37c09cbc0 con 0x7fc388101940 2024-04-25T11:42:39.656 INFO:teuthology.orchestra.run.smithi083.stdout:167503724596 2024-04-25T11:42:39.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.671+0000 7fc38d133700 1 -- 172.21.15.83:0/2946031295 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc3700a3cd0 msgr2=0x7fc3700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:39.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.671+0000 7fc38d133700 1 --2- 172.21.15.83:0/2946031295 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc3700a3cd0 0x7fc3700a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc37800b5c0 tx=0x7fc378005960 comp rx=0 tx=0).stop 2024-04-25T11:42:39.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.671+0000 7fc38d133700 1 -- 172.21.15.83:0/2946031295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388101940 msgr2=0x7fc3881991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:39.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.671+0000 7fc38d133700 1 --2- 172.21.15.83:0/2946031295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388101940 0x7fc3881991e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00b530 tx=0x7fc37c00b840 comp rx=0 tx=0).stop 2024-04-25T11:42:39.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.671+0000 7fc38d133700 1 -- 172.21.15.83:0/2946031295 shutdown_connections 2024-04-25T11:42:39.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.671+0000 7fc38d133700 1 --2- 172.21.15.83:0/2946031295 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc3700a3cd0 0x7fc3700a6190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:39.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.672+0000 7fc38d133700 1 --2- 172.21.15.83:0/2946031295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc388107990 0x7fc38819da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:39.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.672+0000 7fc38d133700 1 --2- 172.21.15.83:0/2946031295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc388101940 0x7fc3881991e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:39.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.672+0000 7fc38d133700 1 --2- 172.21.15.83:0/2946031295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc388102300 0x7fc388199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:39.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.672+0000 7fc38d133700 1 -- 172.21.15.83:0/2946031295 >> 172.21.15.83:0/2946031295 conn(0x7fc3880788f0 msgr2=0x7fc38806ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:39.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.672+0000 7fc38d133700 1 -- 172.21.15.83:0/2946031295 shutdown_connections 2024-04-25T11:42:39.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:39.672+0000 7fc38d133700 1 -- 172.21.15.83:0/2946031295 wait complete. 2024-04-25T11:42:40.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:40 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2946031295' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T11:42:40.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:40 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2946031295' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T11:42:40.490 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149755 got 111669149759 for osd.3 2024-04-25T11:42:40.491 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:40.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:40 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2946031295' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T11:42:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.030+0000 7fb99b1fb700 1 -- 172.21.15.83:0/35655004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb994113c10 msgr2=0x7fb994116000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.030+0000 7fb99b1fb700 1 --2- 172.21.15.83:0/35655004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb994113c10 0x7fb994116000 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00b0a0 tx=0x7fb98c00b3b0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.030+0000 7fb99b1fb700 1 -- 172.21.15.83:0/35655004 shutdown_connections 2024-04-25T11:42:41.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.030+0000 7fb99b1fb700 1 --2- 172.21.15.83:0/35655004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb994113c10 0x7fb994116000 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.030+0000 7fb99b1fb700 1 --2- 172.21.15.83:0/35655004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994074c70 0x7fb994075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.030+0000 7fb99b1fb700 1 --2- 172.21.15.83:0/35655004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb994075590 0x7fb9941136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.030+0000 7fb99b1fb700 1 -- 172.21.15.83:0/35655004 >> 172.21.15.83:0/35655004 conn(0x7fb994079bf0 msgr2=0x7fb99407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.030+0000 7fb99b1fb700 1 -- 172.21.15.83:0/35655004 shutdown_connections 2024-04-25T11:42:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.030+0000 7fb99b1fb700 1 -- 172.21.15.83:0/35655004 wait complete. 2024-04-25T11:42:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb99b1fb700 1 Processor -- start 2024-04-25T11:42:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb99b1fb700 1 -- start start 2024-04-25T11:42:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb99b1fb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb994074c70 0x7fb99419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb99b1fb700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb994075590 0x7fb99419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb99b1fb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994113c10 0x7fb9941a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb99b1fb700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9941191f0 con 0x7fb994075590 2024-04-25T11:42:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb99b1fb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb994119070 con 0x7fb994113c10 2024-04-25T11:42:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb99b1fb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb994119370 con 0x7fb994074c70 2024-04-25T11:42:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb993fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb994075590 0x7fb99419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:41.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb993fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb994075590 0x7fb99419fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51292/0 (socket says 172.21.15.83:51292) 2024-04-25T11:42:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb993fff700 1 -- 172.21.15.83:0/2736054272 learned_addr learned my addr 172.21.15.83:0/2736054272 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb998f97700 1 --2- 172.21.15.83:0/2736054272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb994074c70 0x7fb99419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb999798700 1 --2- 172.21.15.83:0/2736054272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994113c10 0x7fb9941a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb993fff700 1 -- 172.21.15.83:0/2736054272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb994074c70 msgr2=0x7fb99419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb993fff700 1 --2- 172.21.15.83:0/2736054272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb994074c70 0x7fb99419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb993fff700 1 -- 172.21.15.83:0/2736054272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994113c10 msgr2=0x7fb9941a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb993fff700 1 --2- 172.21.15.83:0/2736054272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994113c10 0x7fb9941a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.031+0000 7fb993fff700 1 -- 172.21.15.83:0/2736054272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9941a4620 con 0x7fb994075590 2024-04-25T11:42:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.032+0000 7fb993fff700 1 --2- 172.21.15.83:0/2736054272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb994075590 0x7fb99419fba0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb98000d980 tx=0x7fb98000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.032+0000 7fb991ffb700 1 -- 172.21.15.83:0/2736054272 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb980016070 con 0x7fb994075590 2024-04-25T11:42:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.032+0000 7fb991ffb700 1 -- 172.21.15.83:0/2736054272 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb980004830 con 0x7fb994075590 2024-04-25T11:42:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.032+0000 7fb991ffb700 1 -- 172.21.15.83:0/2736054272 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb980005540 con 0x7fb994075590 2024-04-25T11:42:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.032+0000 7fb99b1fb700 1 -- 172.21.15.83:0/2736054272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9941a4910 con 0x7fb994075590 2024-04-25T11:42:41.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.032+0000 7fb99b1fb700 1 -- 172.21.15.83:0/2736054272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9941ac190 con 0x7fb994075590 2024-04-25T11:42:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.033+0000 7fb99b1fb700 1 -- 172.21.15.83:0/2736054272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb994069730 con 0x7fb994075590 2024-04-25T11:42:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.034+0000 7fb991ffb700 1 -- 172.21.15.83:0/2736054272 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb980004220 con 0x7fb994075590 2024-04-25T11:42:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.035+0000 7fb991ffb700 1 --2- 172.21.15.83:0/2736054272 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb9840a3d50 0x7fb9840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.035+0000 7fb998f97700 1 --2- 172.21.15.83:0/2736054272 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb9840a3d50 0x7fb9840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:41.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.035+0000 7fb991ffb700 1 -- 172.21.15.83:0/2736054272 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb9800d0050 con 0x7fb994075590 2024-04-25T11:42:41.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.035+0000 7fb998f97700 1 --2- 172.21.15.83:0/2736054272 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb9840a3d50 0x7fb9840a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb98800ac50 tx=0x7fb98800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:41.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.039+0000 7fb991ffb700 1 -- 172.21.15.83:0/2736054272 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb98001f020 con 0x7fb994075590 2024-04-25T11:42:41.082 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:41 smithi083 ceph-mon[29330]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T11:42:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.264+0000 7fb99b1fb700 1 -- 172.21.15.83:0/2736054272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fb9941ac390 con 0x7fb994075590 2024-04-25T11:42:41.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.264+0000 7fb991ffb700 1 -- 172.21.15.83:0/2736054272 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb980022350 con 0x7fb994075590 2024-04-25T11:42:41.251 INFO:teuthology.orchestra.run.smithi083.stdout:292057776162 2024-04-25T11:42:41.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.266+0000 7fb99b1fb700 1 -- 172.21.15.83:0/2736054272 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb9840a3d50 msgr2=0x7fb9840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:41.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.266+0000 7fb99b1fb700 1 --2- 172.21.15.83:0/2736054272 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb9840a3d50 0x7fb9840a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb98800ac50 tx=0x7fb98800a380 comp rx=0 tx=0).stop 2024-04-25T11:42:41.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.266+0000 7fb99b1fb700 1 -- 172.21.15.83:0/2736054272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb994075590 msgr2=0x7fb99419fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:41.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.266+0000 7fb99b1fb700 1 --2- 172.21.15.83:0/2736054272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb994075590 0x7fb99419fba0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb98000d980 tx=0x7fb98000dc90 comp rx=0 tx=0).stop 2024-04-25T11:42:41.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.266+0000 7fb99b1fb700 1 -- 172.21.15.83:0/2736054272 shutdown_connections 2024-04-25T11:42:41.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.267+0000 7fb99b1fb700 1 --2- 172.21.15.83:0/2736054272 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fb9840a3d50 0x7fb9840a6210 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.267+0000 7fb99b1fb700 1 --2- 172.21.15.83:0/2736054272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb994113c10 0x7fb9941a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.267+0000 7fb99b1fb700 1 --2- 172.21.15.83:0/2736054272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb994074c70 0x7fb99419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.267+0000 7fb99b1fb700 1 --2- 172.21.15.83:0/2736054272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb994075590 0x7fb99419fba0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.267+0000 7fb99b1fb700 1 -- 172.21.15.83:0/2736054272 >> 172.21.15.83:0/2736054272 conn(0x7fb994079bf0 msgr2=0x7fb994112960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.267+0000 7fb99b1fb700 1 -- 172.21.15.83:0/2736054272 shutdown_connections 2024-04-25T11:42:41.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:41.267+0000 7fb99b1fb700 1 -- 172.21.15.83:0/2736054272 wait complete. 2024-04-25T11:42:41.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:41 smithi111 ceph-mon[32784]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T11:42:41.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:41 smithi094 ceph-mon[33032]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T11:42:42.250 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:42 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2736054272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T11:42:42.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:42 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2736054272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T11:42:42.326 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:42 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2736054272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T11:42:42.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.577+0000 7f91c3d46700 1 -- 172.21.15.83:0/2854078125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91bc102330 msgr2=0x7f91bc10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.577+0000 7f91c3d46700 1 --2- 172.21.15.83:0/2854078125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91bc102330 0x7f91bc10e7f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f91b400b0a0 tx=0x7f91b400b3b0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.577+0000 7f91c3d46700 1 -- 172.21.15.83:0/2854078125 shutdown_connections 2024-04-25T11:42:42.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.577+0000 7f91c3d46700 1 --2- 172.21.15.83:0/2854078125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91bc102330 0x7f91bc10e7f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.577+0000 7f91c3d46700 1 --2- 172.21.15.83:0/2854078125 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91bc107920 0x7f91bc107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.577+0000 7f91c3d46700 1 --2- 172.21.15.83:0/2854078125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91bc101970 0x7f91bc101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.577+0000 7f91c3d46700 1 -- 172.21.15.83:0/2854078125 >> 172.21.15.83:0/2854078125 conn(0x7f91bc078910 msgr2=0x7f91bc078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:42.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.578+0000 7f91c3d46700 1 -- 172.21.15.83:0/2854078125 shutdown_connections 2024-04-25T11:42:42.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.578+0000 7f91c3d46700 1 -- 172.21.15.83:0/2854078125 wait complete. 2024-04-25T11:42:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.578+0000 7f91c3d46700 1 Processor -- start 2024-04-25T11:42:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.578+0000 7f91c3d46700 1 -- start start 2024-04-25T11:42:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.578+0000 7f91c3d46700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91bc101970 0x7f91bc199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.578+0000 7f91c3d46700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91bc102330 0x7f91bc1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.578+0000 7f91c3d46700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91bc107920 0x7f91bc19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.578+0000 7f91c3d46700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc111b00 con 0x7f91bc102330 2024-04-25T11:42:42.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.578+0000 7f91c3d46700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc111980 con 0x7f91bc107920 2024-04-25T11:42:42.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.578+0000 7f91c3d46700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc111c80 con 0x7f91bc101970 2024-04-25T11:42:42.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.578+0000 7f91c12e1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91bc102330 0x7f91bc1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.578+0000 7f91c12e1700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91bc102330 0x7f91bc1995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37012/0 (socket says 172.21.15.83:37012) 2024-04-25T11:42:42.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.579+0000 7f91c12e1700 1 -- 172.21.15.83:0/3341511315 learned_addr learned my addr 172.21.15.83:0/3341511315 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:42.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.579+0000 7f91c22e3700 1 --2- 172.21.15.83:0/3341511315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91bc107920 0x7f91bc19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.579+0000 7f91c1ae2700 1 --2- 172.21.15.83:0/3341511315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91bc101970 0x7f91bc199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.579+0000 7f91c12e1700 1 -- 172.21.15.83:0/3341511315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91bc101970 msgr2=0x7f91bc199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.579+0000 7f91c12e1700 1 --2- 172.21.15.83:0/3341511315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91bc101970 0x7f91bc199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.579+0000 7f91c12e1700 1 -- 172.21.15.83:0/3341511315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91bc107920 msgr2=0x7f91bc19d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.579+0000 7f91c12e1700 1 --2- 172.21.15.83:0/3341511315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91bc107920 0x7f91bc19d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.579+0000 7f91c12e1700 1 -- 172.21.15.83:0/3341511315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91bc19e090 con 0x7f91bc102330 2024-04-25T11:42:42.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.579+0000 7f91c12e1700 1 --2- 172.21.15.83:0/3341511315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91bc102330 0x7f91bc1995c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00c9b0 tx=0x7f91ac00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:42.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.579+0000 7f91b2ffd700 1 -- 172.21.15.83:0/3341511315 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac007b00 con 0x7f91bc102330 2024-04-25T11:42:42.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.579+0000 7f91c3d46700 1 -- 172.21.15.83:0/3341511315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91bc19e380 con 0x7f91bc102330 2024-04-25T11:42:42.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.579+0000 7f91b2ffd700 1 -- 172.21.15.83:0/3341511315 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91ac007ca0 con 0x7f91bc102330 2024-04-25T11:42:42.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.579+0000 7f91b2ffd700 1 -- 172.21.15.83:0/3341511315 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac018730 con 0x7f91bc102330 2024-04-25T11:42:42.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.579+0000 7f91c3d46700 1 -- 172.21.15.83:0/3341511315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91bc1a5c00 con 0x7f91bc102330 2024-04-25T11:42:42.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.580+0000 7f91c3d46700 1 -- 172.21.15.83:0/3341511315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91bc069730 con 0x7f91bc102330 2024-04-25T11:42:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.581+0000 7f91b2ffd700 1 -- 172.21.15.83:0/3341511315 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f91ac00f450 con 0x7f91bc102330 2024-04-25T11:42:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.582+0000 7f91b2ffd700 1 --2- 172.21.15.83:0/3341511315 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f91a80a3cd0 0x7f91a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.582+0000 7f91c1ae2700 1 --2- 172.21.15.83:0/3341511315 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f91a80a3cd0 0x7f91a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.582+0000 7f91b2ffd700 1 -- 172.21.15.83:0/3341511315 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f91ac0d0790 con 0x7f91bc102330 2024-04-25T11:42:42.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.582+0000 7f91c1ae2700 1 --2- 172.21.15.83:0/3341511315 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f91a80a3cd0 0x7f91a80a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f91b8005fd0 tx=0x7f91b8005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:42.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.586+0000 7f91b2ffd700 1 -- 172.21.15.83:0/3341511315 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f91ac014030 con 0x7f91bc102330 2024-04-25T11:42:42.750 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:42.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.794+0000 7f91c3d46700 1 -- 172.21.15.83:0/3341511315 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f91bc19a3e0 con 0x7f91bc102330 2024-04-25T11:42:42.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.794+0000 7f91b2ffd700 1 -- 172.21.15.83:0/3341511315 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f91ac09d250 con 0x7f91bc102330 2024-04-25T11:42:42.781 INFO:teuthology.orchestra.run.smithi083.stdout:55834574920 2024-04-25T11:42:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.796+0000 7f91c3d46700 1 -- 172.21.15.83:0/3341511315 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f91a80a3cd0 msgr2=0x7f91a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.796+0000 7f91c3d46700 1 --2- 172.21.15.83:0/3341511315 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f91a80a3cd0 0x7f91a80a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f91b8005fd0 tx=0x7f91b8005ce0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.796+0000 7f91c3d46700 1 -- 172.21.15.83:0/3341511315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91bc102330 msgr2=0x7f91bc1995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.796+0000 7f91c3d46700 1 --2- 172.21.15.83:0/3341511315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91bc102330 0x7f91bc1995c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00c9b0 tx=0x7f91ac00cd70 comp rx=0 tx=0).stop 2024-04-25T11:42:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.796+0000 7f91c3d46700 1 -- 172.21.15.83:0/3341511315 shutdown_connections 2024-04-25T11:42:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.796+0000 7f91c3d46700 1 --2- 172.21.15.83:0/3341511315 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f91a80a3cd0 0x7f91a80a6190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.796+0000 7f91c3d46700 1 --2- 172.21.15.83:0/3341511315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91bc107920 0x7f91bc19d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.796+0000 7f91c3d46700 1 --2- 172.21.15.83:0/3341511315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f91bc101970 0x7f91bc199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.796+0000 7f91c3d46700 1 --2- 172.21.15.83:0/3341511315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91bc102330 0x7f91bc1995c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.796+0000 7f91c3d46700 1 -- 172.21.15.83:0/3341511315 >> 172.21.15.83:0/3341511315 conn(0x7f91bc078910 msgr2=0x7f91bc06b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.796+0000 7f91c3d46700 1 -- 172.21.15.83:0/3341511315 shutdown_connections 2024-04-25T11:42:42.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.796+0000 7f91c3d46700 1 -- 172.21.15.83:0/3341511315 wait complete. 2024-04-25T11:42:42.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.989+0000 7f362fdaf700 1 -- 172.21.15.83:0/1294187939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281025e0 msgr2=0x7f362810e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.989+0000 7f362fdaf700 1 --2- 172.21.15.83:0/1294187939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281025e0 0x7f362810e7d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f362000b0a0 tx=0x7f362000b3b0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.989+0000 7f362fdaf700 1 -- 172.21.15.83:0/1294187939 shutdown_connections 2024-04-25T11:42:42.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.989+0000 7f362fdaf700 1 --2- 172.21.15.83:0/1294187939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281025e0 0x7f362810e7d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.989+0000 7f362fdaf700 1 --2- 172.21.15.83:0/1294187939 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3628106940 0x7f3628106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.989+0000 7f362fdaf700 1 --2- 172.21.15.83:0/1294187939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3628101c20 0x7f36281020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.989+0000 7f362fdaf700 1 -- 172.21.15.83:0/1294187939 >> 172.21.15.83:0/1294187939 conn(0x7f3628078830 msgr2=0x7f3628078c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:42.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.989+0000 7f362fdaf700 1 -- 172.21.15.83:0/1294187939 shutdown_connections 2024-04-25T11:42:42.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.989+0000 7f362fdaf700 1 -- 172.21.15.83:0/1294187939 wait complete. 2024-04-25T11:42:42.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.989+0000 7f362fdaf700 1 Processor -- start 2024-04-25T11:42:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362fdaf700 1 -- start start 2024-04-25T11:42:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362fdaf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3628101c20 0x7f3628198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362fdaf700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281025e0 0x7f3628198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362fdaf700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3628106940 0x7f362819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362fdaf700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3628111b80 con 0x7f36281025e0 2024-04-25T11:42:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362fdaf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3628111a00 con 0x7f3628101c20 2024-04-25T11:42:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362fdaf700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3628111d00 con 0x7f3628106940 2024-04-25T11:42:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362d34a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281025e0 0x7f3628198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362d34a700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281025e0 0x7f3628198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37022/0 (socket says 172.21.15.83:37022) 2024-04-25T11:42:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362d34a700 1 -- 172.21.15.83:0/2925313055 learned_addr learned my addr 172.21.15.83:0/2925313055 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362e34c700 1 --2- 172.21.15.83:0/2925313055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3628106940 0x7f362819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362d34a700 1 -- 172.21.15.83:0/2925313055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3628106940 msgr2=0x7f362819c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362d34a700 1 --2- 172.21.15.83:0/2925313055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3628106940 0x7f362819c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362db4b700 1 --2- 172.21.15.83:0/2925313055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3628101c20 0x7f3628198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362d34a700 1 -- 172.21.15.83:0/2925313055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3628101c20 msgr2=0x7f3628198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:42.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362d34a700 1 --2- 172.21.15.83:0/2925313055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3628101c20 0x7f3628198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:42.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362d34a700 1 -- 172.21.15.83:0/2925313055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362819d010 con 0x7f36281025e0 2024-04-25T11:42:42.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362e34c700 1 --2- 172.21.15.83:0/2925313055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3628106940 0x7f362819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:42.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f362d34a700 1 --2- 172.21.15.83:0/2925313055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281025e0 0x7f3628198590 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f361800c920 tx=0x7f361800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:42.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.990+0000 7f361effd700 1 -- 172.21.15.83:0/2925313055 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618007b00 con 0x7f36281025e0 2024-04-25T11:42:42.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.991+0000 7f361effd700 1 -- 172.21.15.83:0/2925313055 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3618004ce0 con 0x7f36281025e0 2024-04-25T11:42:42.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.991+0000 7f361effd700 1 -- 172.21.15.83:0/2925313055 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618005650 con 0x7f36281025e0 2024-04-25T11:42:42.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.991+0000 7f362fdaf700 1 -- 172.21.15.83:0/2925313055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362819d300 con 0x7f36281025e0 2024-04-25T11:42:42.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.991+0000 7f362fdaf700 1 -- 172.21.15.83:0/2925313055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36281a4b80 con 0x7f36281025e0 2024-04-25T11:42:42.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.991+0000 7f362fdaf700 1 -- 172.21.15.83:0/2925313055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36281030b0 con 0x7f36281025e0 2024-04-25T11:42:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.993+0000 7f361effd700 1 -- 172.21.15.83:0/2925313055 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3618004750 con 0x7f36281025e0 2024-04-25T11:42:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.993+0000 7f361effd700 1 --2- 172.21.15.83:0/2925313055 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f36140a3cd0 0x7f36140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.993+0000 7f362db4b700 1 --2- 172.21.15.83:0/2925313055 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f36140a3cd0 0x7f36140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.993+0000 7f361effd700 1 -- 172.21.15.83:0/2925313055 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f36180d0510 con 0x7f36281025e0 2024-04-25T11:42:42.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.994+0000 7f362db4b700 1 --2- 172.21.15.83:0/2925313055 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f36140a3cd0 0x7f36140a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3624006fd0 tx=0x7f3624008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:42.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:42.997+0000 7f361effd700 1 -- 172.21.15.83:0/2925313055 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3618014030 con 0x7f36281025e0 2024-04-25T11:42:43.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:43 smithi083 ceph-mon[29330]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:42:43.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:43 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3341511315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T11:42:43.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:43.212+0000 7f362fdaf700 1 -- 172.21.15.83:0/2925313055 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f362802df80 con 0x7f36281025e0 2024-04-25T11:42:43.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:43.212+0000 7f361effd700 1 -- 172.21.15.83:0/2925313055 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f361809cf40 con 0x7f36281025e0 2024-04-25T11:42:43.199 INFO:teuthology.orchestra.run.smithi083.stdout:201863462961 2024-04-25T11:42:43.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:43.214+0000 7f362fdaf700 1 -- 172.21.15.83:0/2925313055 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f36140a3cd0 msgr2=0x7f36140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:43.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:43.214+0000 7f362fdaf700 1 --2- 172.21.15.83:0/2925313055 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f36140a3cd0 0x7f36140a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3624006fd0 tx=0x7f3624008040 comp rx=0 tx=0).stop 2024-04-25T11:42:43.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:43.214+0000 7f362fdaf700 1 -- 172.21.15.83:0/2925313055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281025e0 msgr2=0x7f3628198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:43.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:43.214+0000 7f362fdaf700 1 --2- 172.21.15.83:0/2925313055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281025e0 0x7f3628198590 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f361800c920 tx=0x7f361800cce0 comp rx=0 tx=0).stop 2024-04-25T11:42:43.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:43.214+0000 7f362fdaf700 1 -- 172.21.15.83:0/2925313055 shutdown_connections 2024-04-25T11:42:43.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:43.214+0000 7f362fdaf700 1 --2- 172.21.15.83:0/2925313055 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f36140a3cd0 0x7f36140a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:43.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:43.214+0000 7f362fdaf700 1 --2- 172.21.15.83:0/2925313055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3628106940 0x7f362819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:43.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:43.214+0000 7f362fdaf700 1 --2- 172.21.15.83:0/2925313055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3628101c20 0x7f3628198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:43.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:43.214+0000 7f362fdaf700 1 --2- 172.21.15.83:0/2925313055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36281025e0 0x7f3628198590 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:43.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:43.215+0000 7f362fdaf700 1 -- 172.21.15.83:0/2925313055 >> 172.21.15.83:0/2925313055 conn(0x7f3628078830 msgr2=0x7f36280ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:43.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:43.215+0000 7f362fdaf700 1 -- 172.21.15.83:0/2925313055 shutdown_connections 2024-04-25T11:42:43.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:43.215+0000 7f362fdaf700 1 -- 172.21.15.83:0/2925313055 wait complete. 2024-04-25T11:42:43.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:43 smithi111 ceph-mon[32784]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:42:43.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:43 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3341511315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T11:42:43.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:43 smithi094 ceph-mon[33032]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:42:43.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:43 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3341511315' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T11:42:43.565 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776157 got 292057776162 for osd.9 2024-04-25T11:42:43.565 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:43.728 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:44.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:44 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2925313055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T11:42:44.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:44 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2925313055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T11:42:44.340 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462956 got 201863462961 for osd.6 2024-04-25T11:42:44.340 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:44.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:44 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2925313055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T11:42:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 -- 172.21.15.83:0/4281433522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea78101930 msgr2=0x7fea78101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 --2- 172.21.15.83:0/4281433522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea78101930 0x7fea78101db0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fea64009990 tx=0x7fea64009ca0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 -- 172.21.15.83:0/4281433522 shutdown_connections 2024-04-25T11:42:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 --2- 172.21.15.83:0/4281433522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea781022f0 0x7fea7810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 --2- 172.21.15.83:0/4281433522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea78107980 0x7fea78107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 --2- 172.21.15.83:0/4281433522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea78101930 0x7fea78101db0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 -- 172.21.15.83:0/4281433522 >> 172.21.15.83:0/4281433522 conn(0x7fea780788b0 msgr2=0x7fea78078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 -- 172.21.15.83:0/4281433522 shutdown_connections 2024-04-25T11:42:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 -- 172.21.15.83:0/4281433522 wait complete. 2024-04-25T11:42:44.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 Processor -- start 2024-04-25T11:42:44.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 -- start start 2024-04-25T11:42:44.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea78101930 0x7fea781990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea781022f0 0x7fea78199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea78107980 0x7fea7819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fea78111ad0 con 0x7fea781022f0 2024-04-25T11:42:44.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fea78111950 con 0x7fea78101930 2024-04-25T11:42:44.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.900+0000 7fea7ed64700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fea78111c50 con 0x7fea78107980 2024-04-25T11:42:44.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea77fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea781022f0 0x7fea78199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea7cb00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea78101930 0x7fea781990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea7d301700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea78107980 0x7fea7819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea7cb00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea78101930 0x7fea781990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.83:39464/0 (socket says 172.21.15.83:39464) 2024-04-25T11:42:44.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea7cb00700 1 -- 172.21.15.83:0/1771683884 learned_addr learned my addr 172.21.15.83:0/1771683884 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:44.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea77fff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea781022f0 0x7fea78199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37082/0 (socket says 172.21.15.83:37082) 2024-04-25T11:42:44.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea7cb00700 1 -- 172.21.15.83:0/1771683884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea78107980 msgr2=0x7fea7819d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea7cb00700 1 --2- 172.21.15.83:0/1771683884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea78107980 0x7fea7819d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea7cb00700 1 -- 172.21.15.83:0/1771683884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea781022f0 msgr2=0x7fea78199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea7cb00700 1 --2- 172.21.15.83:0/1771683884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea781022f0 0x7fea78199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea7cb00700 1 -- 172.21.15.83:0/1771683884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7819e0e0 con 0x7fea78101930 2024-04-25T11:42:44.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea7cb00700 1 --2- 172.21.15.83:0/1771683884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea78101930 0x7fea781990d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00e9a0 tx=0x7fea6c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea75ffb700 1 -- 172.21.15.83:0/1771683884 <== mon.2 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c00ca10 con 0x7fea78101930 2024-04-25T11:42:44.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea7ed64700 1 -- 172.21.15.83:0/1771683884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7819e3d0 con 0x7fea78101930 2024-04-25T11:42:44.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea75ffb700 1 -- 172.21.15.83:0/1771683884 <== mon.2 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea6c00cbb0 con 0x7fea78101930 2024-04-25T11:42:44.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.901+0000 7fea75ffb700 1 -- 172.21.15.83:0/1771683884 <== mon.2 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c018680 con 0x7fea78101930 2024-04-25T11:42:44.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.902+0000 7fea7ed64700 1 -- 172.21.15.83:0/1771683884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea781a5c50 con 0x7fea78101930 2024-04-25T11:42:44.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.903+0000 7fea7ed64700 1 -- 172.21.15.83:0/1771683884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea78069730 con 0x7fea78101930 2024-04-25T11:42:44.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.903+0000 7fea75ffb700 1 -- 172.21.15.83:0/1771683884 <== mon.2 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fea6c010430 con 0x7fea78101930 2024-04-25T11:42:44.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.904+0000 7fea75ffb700 1 --2- 172.21.15.83:0/1771683884 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fea680a3cd0 0x7fea680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.904+0000 7fea77fff700 1 --2- 172.21.15.83:0/1771683884 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fea680a3cd0 0x7fea680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.905+0000 7fea75ffb700 1 -- 172.21.15.83:0/1771683884 <== mon.2 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fea6c0d05c0 con 0x7fea78101930 2024-04-25T11:42:44.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.905+0000 7fea77fff700 1 --2- 172.21.15.83:0/1771683884 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fea680a3cd0 0x7fea680a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fea6400b2e0 tx=0x7fea64005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.909+0000 7fea75ffb700 1 -- 172.21.15.83:0/1771683884 <== mon.2 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea6c09cef0 con 0x7fea78101930 2024-04-25T11:42:44.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.804+0000 7f58f6a69700 1 -- 172.21.15.83:0/4148848126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f0107990 msgr2=0x7f58f0107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.804+0000 7f58f6a69700 1 --2- 172.21.15.83:0/4148848126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f0107990 0x7f58f0107d70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009990 tx=0x7f58e4009ca0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.804+0000 7f58f6a69700 1 -- 172.21.15.83:0/4148848126 shutdown_connections 2024-04-25T11:42:44.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.804+0000 7f58f6a69700 1 --2- 172.21.15.83:0/4148848126 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f0102300 0x7f58f010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.804+0000 7f58f6a69700 1 --2- 172.21.15.83:0/4148848126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f0107990 0x7f58f0107d70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.804+0000 7f58f6a69700 1 --2- 172.21.15.83:0/4148848126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f0101940 0x7f58f0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.804+0000 7f58f6a69700 1 -- 172.21.15.83:0/4148848126 >> 172.21.15.83:0/4148848126 conn(0x7f58f00788f0 msgr2=0x7f58f0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:44.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.804+0000 7f58f6a69700 1 -- 172.21.15.83:0/4148848126 shutdown_connections 2024-04-25T11:42:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.804+0000 7f58f6a69700 1 -- 172.21.15.83:0/4148848126 wait complete. 2024-04-25T11:42:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.805+0000 7f58f6a69700 1 Processor -- start 2024-04-25T11:42:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.805+0000 7f58f6a69700 1 -- start start 2024-04-25T11:42:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.805+0000 7f58f6a69700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f0101940 0x7f58f0199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.805+0000 7f58f6a69700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f0102300 0x7f58f01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.805+0000 7f58f6a69700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f0107990 0x7f58f019da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.805+0000 7f58f6a69700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0111b70 con 0x7f58f0107990 2024-04-25T11:42:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.805+0000 7f58f6a69700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f58f01119f0 con 0x7f58f0102300 2024-04-25T11:42:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.805+0000 7f58f6a69700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f58f0111cf0 con 0x7f58f0101940 2024-04-25T11:42:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.805+0000 7f58f5006700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f0107990 0x7f58f019da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58f5006700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f0107990 0x7f58f019da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37052/0 (socket says 172.21.15.83:37052) 2024-04-25T11:42:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58f5006700 1 -- 172.21.15.83:0/3688676503 learned_addr learned my addr 172.21.15.83:0/3688676503 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:44.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58effff700 1 --2- 172.21.15.83:0/3688676503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f0102300 0x7f58f01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58f4805700 1 --2- 172.21.15.83:0/3688676503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f0101940 0x7f58f0199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58f5006700 1 -- 172.21.15.83:0/3688676503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f0101940 msgr2=0x7f58f0199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58f5006700 1 --2- 172.21.15.83:0/3688676503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f0101940 0x7f58f0199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58f5006700 1 -- 172.21.15.83:0/3688676503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f0102300 msgr2=0x7f58f01996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58f5006700 1 --2- 172.21.15.83:0/3688676503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f0102300 0x7f58f01996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58f5006700 1 -- 172.21.15.83:0/3688676503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f019e180 con 0x7f58f0107990 2024-04-25T11:42:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58effff700 1 --2- 172.21.15.83:0/3688676503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f0102300 0x7f58f01996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:42:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58f5006700 1 --2- 172.21.15.83:0/3688676503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f0107990 0x7f58f019da40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f58e800b970 tx=0x7f58e800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58edffb700 1 -- 172.21.15.83:0/3688676503 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e800d860 con 0x7f58f0107990 2024-04-25T11:42:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58edffb700 1 -- 172.21.15.83:0/3688676503 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e800e430 con 0x7f58f0107990 2024-04-25T11:42:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58edffb700 1 -- 172.21.15.83:0/3688676503 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8017660 con 0x7f58f0107990 2024-04-25T11:42:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58f6a69700 1 -- 172.21.15.83:0/3688676503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f019e470 con 0x7f58f0107990 2024-04-25T11:42:44.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.806+0000 7f58f6a69700 1 -- 172.21.15.83:0/3688676503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f01a5cf0 con 0x7f58f0107990 2024-04-25T11:42:44.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.808+0000 7f58f6a69700 1 -- 172.21.15.83:0/3688676503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f0069730 con 0x7f58f0107990 2024-04-25T11:42:44.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.808+0000 7f58edffb700 1 -- 172.21.15.83:0/3688676503 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f58e800da00 con 0x7f58f0107990 2024-04-25T11:42:44.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.809+0000 7f58edffb700 1 --2- 172.21.15.83:0/3688676503 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f58e00a3cd0 0x7f58e00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:44.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.809+0000 7f58f4805700 1 --2- 172.21.15.83:0/3688676503 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f58e00a3cd0 0x7f58e00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:44.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.809+0000 7f58edffb700 1 -- 172.21.15.83:0/3688676503 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f58e80cffc0 con 0x7f58f0107990 2024-04-25T11:42:44.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.809+0000 7f58f4805700 1 --2- 172.21.15.83:0/3688676503 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f58e00a3cd0 0x7f58e00a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f58e4006010 tx=0x7f58e400b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:44.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:44.814+0000 7f58edffb700 1 -- 172.21.15.83:0/3688676503 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58e8013030 con 0x7f58f0107990 2024-04-25T11:42:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.026+0000 7f58f6a69700 1 -- 172.21.15.83:0/3688676503 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f58f01a5ec0 con 0x7f58f0107990 2024-04-25T11:42:45.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.027+0000 7f58edffb700 1 -- 172.21.15.83:0/3688676503 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f58e809c9f0 con 0x7f58f0107990 2024-04-25T11:42:45.014 INFO:teuthology.orchestra.run.smithi083.stdout:34359738446 2024-04-25T11:42:45.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.029+0000 7f58f6a69700 1 -- 172.21.15.83:0/3688676503 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f58e00a3cd0 msgr2=0x7f58e00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:45.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.029+0000 7f58f6a69700 1 --2- 172.21.15.83:0/3688676503 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f58e00a3cd0 0x7f58e00a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f58e4006010 tx=0x7f58e400b560 comp rx=0 tx=0).stop 2024-04-25T11:42:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.029+0000 7f58f6a69700 1 -- 172.21.15.83:0/3688676503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f0107990 msgr2=0x7f58f019da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.029+0000 7f58f6a69700 1 --2- 172.21.15.83:0/3688676503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f0107990 0x7f58f019da40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f58e800b970 tx=0x7f58e800bc80 comp rx=0 tx=0).stop 2024-04-25T11:42:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.029+0000 7f58f6a69700 1 -- 172.21.15.83:0/3688676503 shutdown_connections 2024-04-25T11:42:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.029+0000 7f58f6a69700 1 --2- 172.21.15.83:0/3688676503 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f58e00a3cd0 0x7f58e00a6190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.029+0000 7f58f6a69700 1 --2- 172.21.15.83:0/3688676503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58f0107990 0x7f58f019da40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.029+0000 7f58f6a69700 1 --2- 172.21.15.83:0/3688676503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58f0101940 0x7f58f0199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.029+0000 7f58f6a69700 1 --2- 172.21.15.83:0/3688676503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58f0102300 0x7f58f01996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.029+0000 7f58f6a69700 1 -- 172.21.15.83:0/3688676503 >> 172.21.15.83:0/3688676503 conn(0x7f58f00788f0 msgr2=0x7f58f0105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:45.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.029+0000 7f58f6a69700 1 -- 172.21.15.83:0/3688676503 shutdown_connections 2024-04-25T11:42:45.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.029+0000 7f58f6a69700 1 -- 172.21.15.83:0/3688676503 wait complete. 2024-04-25T11:42:45.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.123+0000 7fea7ed64700 1 -- 172.21.15.83:0/1771683884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fea7802df80 con 0x7fea78101930 2024-04-25T11:42:45.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.123+0000 7fea75ffb700 1 -- 172.21.15.83:0/1771683884 <== mon.2 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fea6c09c7f0 con 0x7fea78101930 2024-04-25T11:42:45.110 INFO:teuthology.orchestra.run.smithi083.stdout:146028888123 2024-04-25T11:42:45.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.125+0000 7fea7ed64700 1 -- 172.21.15.83:0/1771683884 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fea680a3cd0 msgr2=0x7fea680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:45.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.125+0000 7fea7ed64700 1 --2- 172.21.15.83:0/1771683884 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fea680a3cd0 0x7fea680a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fea6400b2e0 tx=0x7fea64005960 comp rx=0 tx=0).stop 2024-04-25T11:42:45.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.125+0000 7fea7ed64700 1 -- 172.21.15.83:0/1771683884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea78101930 msgr2=0x7fea781990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:45.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.125+0000 7fea7ed64700 1 --2- 172.21.15.83:0/1771683884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea78101930 0x7fea781990d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00e9a0 tx=0x7fea6c00ecb0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.125+0000 7fea7ed64700 1 -- 172.21.15.83:0/1771683884 shutdown_connections 2024-04-25T11:42:45.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.125+0000 7fea7ed64700 1 --2- 172.21.15.83:0/1771683884 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fea680a3cd0 0x7fea680a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.125+0000 7fea7ed64700 1 --2- 172.21.15.83:0/1771683884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea78107980 0x7fea7819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.125+0000 7fea7ed64700 1 --2- 172.21.15.83:0/1771683884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea78101930 0x7fea781990d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.125+0000 7fea7ed64700 1 --2- 172.21.15.83:0/1771683884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fea781022f0 0x7fea78199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:45.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.126+0000 7fea7ed64700 1 -- 172.21.15.83:0/1771683884 >> 172.21.15.83:0/1771683884 conn(0x7fea780788b0 msgr2=0x7fea7806b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:45.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.126+0000 7fea7ed64700 1 -- 172.21.15.83:0/1771683884 shutdown_connections 2024-04-25T11:42:45.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:45.126+0000 7fea7ed64700 1 -- 172.21.15.83:0/1771683884 wait complete. 2024-04-25T11:42:45.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:45 smithi094 ceph-mon[33032]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T11:42:45.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:45 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:42:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:45 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3688676503' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T11:42:45.274 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:45 smithi083 ceph-mon[29330]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T11:42:45.274 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:45 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:42:45.274 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:45 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3688676503' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T11:42:45.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:45 smithi111 ceph-mon[32784]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T11:42:45.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:45 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:42:45.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:45 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3688676503' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T11:42:45.923 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724593 got 167503724596 for osd.5 2024-04-25T11:42:45.924 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:46.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:46 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1771683884' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T11:42:46.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:46 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:46.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:46 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:46.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:46 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:46.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:46 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:46.315 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574916 got 55834574920 for osd.1 2024-04-25T11:42:46.315 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:46.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:46 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1771683884' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T11:42:46.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:46 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:46.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:46 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:46.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:46 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:46.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:46 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:46.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:46 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1771683884' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T11:42:46.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:46 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:46.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:46 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:46.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:46 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:46.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:46 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:46.744 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612632 got 313532612636 for osd.10 2024-04-25T11:42:46.745 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:47.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:47 smithi111 ceph-mon[32784]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T11:42:47.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:47 smithi083 ceph-mon[29330]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T11:42:47.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:47 smithi094 ceph-mon[33032]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T11:42:48.041 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888117 got 146028888123 for osd.4 2024-04-25T11:42:48.041 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:48.689 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738440 got 34359738446 for osd.0 2024-04-25T11:42:48.689 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:49.051 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:49.052 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:49.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:49 smithi111 ceph-mon[32784]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:42:49.326 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:49 smithi083 ceph-mon[29330]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:42:49.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:49 smithi094 ceph-mon[33032]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:42:50.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-25T11:42:50.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-25T11:42:50.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-04-25T11:42:50.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-25T11:42:50.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-25T11:42:50.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-25T11:42:50.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-04-25T11:42:50.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:50 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-25T11:42:50.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-25T11:42:50.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-25T11:42:50.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-04-25T11:42:50.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-25T11:42:50.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-25T11:42:50.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-25T11:42:50.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-04-25T11:42:50.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:50 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-25T11:42:50.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-25T11:42:50.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-25T11:42:50.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-04-25T11:42:50.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-25T11:42:50.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-25T11:42:50.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-25T11:42:50.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-04-25T11:42:50.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:50 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-25T11:42:51.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:51 smithi111 ceph-mon[32784]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T11:42:51.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:51 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-25T11:42:51.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:51 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-25T11:42:51.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:51 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-04-25T11:42:51.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:51 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-25T11:42:51.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:51 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-25T11:42:51.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:51 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-25T11:42:51.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:51 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-04-25T11:42:51.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:51 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-25T11:42:51.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:51 smithi111 ceph-mon[32784]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T11:42:51.383 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:51 smithi083 ceph-mon[29330]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T11:42:51.383 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:51 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-25T11:42:51.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:51 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-25T11:42:51.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:51 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-04-25T11:42:51.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:51 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-25T11:42:51.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:51 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-25T11:42:51.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:51 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-25T11:42:51.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:51 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-04-25T11:42:51.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:51 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-25T11:42:51.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:51 smithi083 ceph-mon[29330]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T11:42:51.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:51 smithi094 ceph-mon[33032]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T11:42:51.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:51 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-25T11:42:51.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:51 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-25T11:42:51.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:51 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-04-25T11:42:51.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:51 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-25T11:42:51.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:51 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-25T11:42:51.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:51 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-25T11:42:51.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:51 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-04-25T11:42:51.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:51 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-25T11:42:51.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:51 smithi094 ceph-mon[33032]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T11:42:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.147+0000 7f4e33b84700 1 -- 172.21.15.83:0/3681116028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c102af0 msgr2=0x7f4e2c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:52.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.147+0000 7f4e33b84700 1 --2- 172.21.15.83:0/3681116028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c102af0 0x7f4e2c102f70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009990 tx=0x7f4e28009ca0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.147+0000 7f4e33b84700 1 -- 172.21.15.83:0/3681116028 shutdown_connections 2024-04-25T11:42:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.147+0000 7f4e33b84700 1 --2- 172.21.15.83:0/3681116028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e2c1034b0 0x7f4e2c10f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.147+0000 7f4e33b84700 1 --2- 172.21.15.83:0/3681116028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e2c108a80 0x7f4e2c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.147+0000 7f4e33b84700 1 --2- 172.21.15.83:0/3681116028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c102af0 0x7f4e2c102f70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.147+0000 7f4e33b84700 1 -- 172.21.15.83:0/3681116028 >> 172.21.15.83:0/3681116028 conn(0x7f4e2c079ad0 msgr2=0x7f4e2c079ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.147+0000 7f4e33b84700 1 -- 172.21.15.83:0/3681116028 shutdown_connections 2024-04-25T11:42:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.147+0000 7f4e33b84700 1 -- 172.21.15.83:0/3681116028 wait complete. 2024-04-25T11:42:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e33b84700 1 Processor -- start 2024-04-25T11:42:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e33b84700 1 -- start start 2024-04-25T11:42:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e33b84700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e2c102af0 0x7f4e2c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e33b84700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e2c1034b0 0x7f4e2c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e33b84700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c108a80 0x7f4e2c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:52.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e33b84700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c112cb0 con 0x7f4e2c108a80 2024-04-25T11:42:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e33b84700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c112b30 con 0x7f4e2c1034b0 2024-04-25T11:42:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e33b84700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c112e30 con 0x7f4e2c102af0 2024-04-25T11:42:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e32121700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c108a80 0x7f4e2c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e32121700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c108a80 0x7f4e2c0757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37100/0 (socket says 172.21.15.83:37100) 2024-04-25T11:42:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e32121700 1 -- 172.21.15.83:0/2378177056 learned_addr learned my addr 172.21.15.83:0/2378177056 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e3111f700 1 --2- 172.21.15.83:0/2378177056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e2c1034b0 0x7f4e2c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e31920700 1 --2- 172.21.15.83:0/2378177056 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e2c102af0 0x7f4e2c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e32121700 1 -- 172.21.15.83:0/2378177056 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e2c102af0 msgr2=0x7f4e2c074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e32121700 1 --2- 172.21.15.83:0/2378177056 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e2c102af0 0x7f4e2c074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e32121700 1 -- 172.21.15.83:0/2378177056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e2c1034b0 msgr2=0x7f4e2c075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e32121700 1 --2- 172.21.15.83:0/2378177056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e2c1034b0 0x7f4e2c075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.148+0000 7f4e32121700 1 -- 172.21.15.83:0/2378177056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c1a7640 con 0x7f4e2c108a80 2024-04-25T11:42:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.149+0000 7f4e32121700 1 --2- 172.21.15.83:0/2378177056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c108a80 0x7f4e2c0757c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400b970 tx=0x7f4e2400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:52.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.149+0000 7f4e22ffd700 1 -- 172.21.15.83:0/2378177056 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e24019070 con 0x7f4e2c108a80 2024-04-25T11:42:52.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.149+0000 7f4e22ffd700 1 -- 172.21.15.83:0/2378177056 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e2400cde0 con 0x7f4e2c108a80 2024-04-25T11:42:52.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.149+0000 7f4e33b84700 1 -- 172.21.15.83:0/2378177056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2c1a7930 con 0x7f4e2c108a80 2024-04-25T11:42:52.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.149+0000 7f4e22ffd700 1 -- 172.21.15.83:0/2378177056 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e24007370 con 0x7f4e2c108a80 2024-04-25T11:42:52.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.149+0000 7f4e33b84700 1 -- 172.21.15.83:0/2378177056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2c1a7e40 con 0x7f4e2c108a80 2024-04-25T11:42:52.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.150+0000 7f4e33b84700 1 -- 172.21.15.83:0/2378177056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e2c069730 con 0x7f4e2c108a80 2024-04-25T11:42:52.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.151+0000 7f4e22ffd700 1 -- 172.21.15.83:0/2378177056 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4e2401c070 con 0x7f4e2c108a80 2024-04-25T11:42:52.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.151+0000 7f4e22ffd700 1 --2- 172.21.15.83:0/2378177056 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4e180a3d50 0x7f4e180a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:52.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.151+0000 7f4e31920700 1 --2- 172.21.15.83:0/2378177056 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4e180a3d50 0x7f4e180a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:52.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.152+0000 7f4e22ffd700 1 -- 172.21.15.83:0/2378177056 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f4e240d0020 con 0x7f4e2c108a80 2024-04-25T11:42:52.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.152+0000 7f4e31920700 1 --2- 172.21.15.83:0/2378177056 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4e180a3d50 0x7f4e180a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c005ea0 tx=0x7f4e1c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:52.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.157+0000 7f4e22ffd700 1 -- 172.21.15.83:0/2378177056 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e24023070 con 0x7f4e2c108a80 2024-04-25T11:42:52.291 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:52 smithi083 ceph-mon[29330]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T11:42:52.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:52 smithi111 ceph-mon[32784]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T11:42:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.368+0000 7f4e33b84700 1 -- 172.21.15.83:0/2378177056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f4e2c02df80 con 0x7f4e2c108a80 2024-04-25T11:42:52.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.369+0000 7f4e22ffd700 1 -- 172.21.15.83:0/2378177056 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f4e2409ca40 con 0x7f4e2c108a80 2024-04-25T11:42:52.355 INFO:teuthology.orchestra.run.smithi083.stdout:347892351001 2024-04-25T11:42:52.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.371+0000 7f4e33b84700 1 -- 172.21.15.83:0/2378177056 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4e180a3d50 msgr2=0x7f4e180a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.371+0000 7f4e33b84700 1 --2- 172.21.15.83:0/2378177056 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4e180a3d50 0x7f4e180a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c005ea0 tx=0x7f4e1c005e30 comp rx=0 tx=0).stop 2024-04-25T11:42:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.371+0000 7f4e33b84700 1 -- 172.21.15.83:0/2378177056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c108a80 msgr2=0x7f4e2c0757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.371+0000 7f4e33b84700 1 --2- 172.21.15.83:0/2378177056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c108a80 0x7f4e2c0757c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400b970 tx=0x7f4e2400bc80 comp rx=0 tx=0).stop 2024-04-25T11:42:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.371+0000 7f4e33b84700 1 -- 172.21.15.83:0/2378177056 shutdown_connections 2024-04-25T11:42:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.371+0000 7f4e33b84700 1 --2- 172.21.15.83:0/2378177056 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f4e180a3d50 0x7f4e180a6210 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.371+0000 7f4e33b84700 1 --2- 172.21.15.83:0/2378177056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e2c108a80 0x7f4e2c0757c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.371+0000 7f4e33b84700 1 --2- 172.21.15.83:0/2378177056 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e2c102af0 0x7f4e2c074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.371+0000 7f4e33b84700 1 --2- 172.21.15.83:0/2378177056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e2c1034b0 0x7f4e2c075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.371+0000 7f4e33b84700 1 -- 172.21.15.83:0/2378177056 >> 172.21.15.83:0/2378177056 conn(0x7f4e2c079ad0 msgr2=0x7f4e2c1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.371+0000 7f4e33b84700 1 -- 172.21.15.83:0/2378177056 shutdown_connections 2024-04-25T11:42:52.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:52.371+0000 7f4e33b84700 1 -- 172.21.15.83:0/2378177056 wait complete. 2024-04-25T11:42:52.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:52 smithi094 ceph-mon[33032]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T11:42:53.076 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350996 got 347892351001 for osd.11 2024-04-25T11:42:53.076 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:53.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.302+0000 7fae072a6700 1 -- 172.21.15.83:0/1338961330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00101030 msgr2=0x7fae001014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.302+0000 7fae072a6700 1 --2- 172.21.15.83:0/1338961330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00101030 0x7fae001014b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009990 tx=0x7fadf0009ca0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.302+0000 7fae072a6700 1 -- 172.21.15.83:0/1338961330 shutdown_connections 2024-04-25T11:42:53.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.302+0000 7fae072a6700 1 --2- 172.21.15.83:0/1338961330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae00101ad0 0x7fae001117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.302+0000 7fae072a6700 1 --2- 172.21.15.83:0/1338961330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae00103950 0x7fae00103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.302+0000 7fae072a6700 1 --2- 172.21.15.83:0/1338961330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00101030 0x7fae001014b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.302+0000 7fae072a6700 1 -- 172.21.15.83:0/1338961330 >> 172.21.15.83:0/1338961330 conn(0x7fae00077fb0 msgr2=0x7fae000ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:53.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.302+0000 7fae072a6700 1 -- 172.21.15.83:0/1338961330 shutdown_connections 2024-04-25T11:42:53.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.302+0000 7fae072a6700 1 -- 172.21.15.83:0/1338961330 wait complete. 2024-04-25T11:42:53.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.303+0000 7fae072a6700 1 Processor -- start 2024-04-25T11:42:53.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.303+0000 7fae072a6700 1 -- start start 2024-04-25T11:42:53.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.303+0000 7fae072a6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae00101030 0x7fae0019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.303+0000 7fae072a6700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00101ad0 0x7fae0019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.303+0000 7fae072a6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae00103950 0x7fae0019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.303+0000 7fae072a6700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae00114ee0 con 0x7fae00101ad0 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.303+0000 7fae072a6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fae00114d60 con 0x7fae00103950 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.303+0000 7fae072a6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fae00115060 con 0x7fae00101030 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.303+0000 7fae04841700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00101ad0 0x7fae0019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.303+0000 7fae04841700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00101ad0 0x7fae0019b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41548/0 (socket says 172.21.15.83:41548) 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.303+0000 7fae04841700 1 -- 172.21.15.83:0/1824677491 learned_addr learned my addr 172.21.15.83:0/1824677491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.303+0000 7fae05042700 1 --2- 172.21.15.83:0/1824677491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae00101030 0x7fae0019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.303+0000 7fae05843700 1 --2- 172.21.15.83:0/1824677491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae00103950 0x7fae0019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.303+0000 7fae04841700 1 -- 172.21.15.83:0/1824677491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae00101030 msgr2=0x7fae0019b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.303+0000 7fae04841700 1 --2- 172.21.15.83:0/1824677491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae00101030 0x7fae0019b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.304+0000 7fae04841700 1 -- 172.21.15.83:0/1824677491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae00103950 msgr2=0x7fae0019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.304+0000 7fae04841700 1 --2- 172.21.15.83:0/1824677491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae00103950 0x7fae0019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.304+0000 7fae04841700 1 -- 172.21.15.83:0/1824677491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae001a0480 con 0x7fae00101ad0 2024-04-25T11:42:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.304+0000 7fae05843700 1 --2- 172.21.15.83:0/1824677491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae00103950 0x7fae0019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.304+0000 7fae04841700 1 --2- 172.21.15.83:0/1824677491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00101ad0 0x7fae0019b9b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fadf0000c00 tx=0x7fadf00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.304+0000 7fadf67fc700 1 -- 172.21.15.83:0/1824677491 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0027070 con 0x7fae00101ad0 2024-04-25T11:42:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.304+0000 7fadf67fc700 1 -- 172.21.15.83:0/1824677491 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadf00040e0 con 0x7fae00101ad0 2024-04-25T11:42:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.304+0000 7fadf67fc700 1 -- 172.21.15.83:0/1824677491 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf000f460 con 0x7fae00101ad0 2024-04-25T11:42:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.304+0000 7fae072a6700 1 -- 172.21.15.83:0/1824677491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae001a0710 con 0x7fae00101ad0 2024-04-25T11:42:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.304+0000 7fae072a6700 1 -- 172.21.15.83:0/1824677491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae001a7ff0 con 0x7fae00101ad0 2024-04-25T11:42:53.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.305+0000 7fae072a6700 1 -- 172.21.15.83:0/1824677491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae00102040 con 0x7fae00101ad0 2024-04-25T11:42:53.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.307+0000 7fadf67fc700 1 -- 172.21.15.83:0/1824677491 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fadf002a070 con 0x7fae00101ad0 2024-04-25T11:42:53.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.307+0000 7fadf67fc700 1 --2- 172.21.15.83:0/1824677491 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fadec0a3d50 0x7fadec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:53.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.307+0000 7fae05042700 1 --2- 172.21.15.83:0/1824677491 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fadec0a3d50 0x7fadec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:53.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.307+0000 7fadf67fc700 1 -- 172.21.15.83:0/1824677491 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fadf00d18d0 con 0x7fae00101ad0 2024-04-25T11:42:53.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.308+0000 7fae05042700 1 --2- 172.21.15.83:0/1824677491 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fadec0a3d50 0x7fadec0a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009a10 tx=0x7fadfc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:53.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.311+0000 7fadf67fc700 1 -- 172.21.15.83:0/1824677491 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fadf0031070 con 0x7fae00101ad0 2024-04-25T11:42:53.297 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:53 smithi083 ceph-mon[29330]: pgmap v319: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-25T11:42:53.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:53 smithi083 ceph-mon[29330]: Health check failed: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-04-25T11:42:53.298 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:53 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2378177056' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T11:42:53.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:53 smithi111 ceph-mon[32784]: pgmap v319: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-25T11:42:53.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:53 smithi111 ceph-mon[32784]: Health check failed: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-04-25T11:42:53.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:53 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2378177056' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T11:42:53.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.517+0000 7fae072a6700 1 -- 172.21.15.83:0/1824677491 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fae0002df80 con 0x7fae00101ad0 2024-04-25T11:42:53.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.517+0000 7fadf67fc700 1 -- 172.21.15.83:0/1824677491 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fadf0013070 con 0x7fae00101ad0 2024-04-25T11:42:53.504 INFO:teuthology.orchestra.run.smithi083.stdout:223338299438 2024-04-25T11:42:53.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.519+0000 7fae072a6700 1 -- 172.21.15.83:0/1824677491 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fadec0a3d50 msgr2=0x7fadec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.519+0000 7fae072a6700 1 --2- 172.21.15.83:0/1824677491 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fadec0a3d50 0x7fadec0a6210 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009a10 tx=0x7fadfc009190 comp rx=0 tx=0).stop 2024-04-25T11:42:53.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.519+0000 7fae072a6700 1 -- 172.21.15.83:0/1824677491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00101ad0 msgr2=0x7fae0019b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:53.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.519+0000 7fae072a6700 1 --2- 172.21.15.83:0/1824677491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00101ad0 0x7fae0019b9b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fadf0000c00 tx=0x7fadf00047b0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.520+0000 7fae072a6700 1 -- 172.21.15.83:0/1824677491 shutdown_connections 2024-04-25T11:42:53.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.520+0000 7fae072a6700 1 --2- 172.21.15.83:0/1824677491 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fadec0a3d50 0x7fadec0a6210 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.520+0000 7fae072a6700 1 --2- 172.21.15.83:0/1824677491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae00103950 0x7fae0019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.520+0000 7fae072a6700 1 --2- 172.21.15.83:0/1824677491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae00101030 0x7fae0019b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.520+0000 7fae072a6700 1 --2- 172.21.15.83:0/1824677491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae00101ad0 0x7fae0019b9b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:53.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.520+0000 7fae072a6700 1 -- 172.21.15.83:0/1824677491 >> 172.21.15.83:0/1824677491 conn(0x7fae00077fb0 msgr2=0x7fae000fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:53.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.520+0000 7fae072a6700 1 -- 172.21.15.83:0/1824677491 shutdown_connections 2024-04-25T11:42:53.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:53.520+0000 7fae072a6700 1 -- 172.21.15.83:0/1824677491 wait complete. 2024-04-25T11:42:53.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:53 smithi094 ceph-mon[33032]: pgmap v319: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-25T11:42:53.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:53 smithi094 ceph-mon[33032]: Health check failed: Reduced data availability: 1 pg inactive, 3 pgs peering (PG_AVAILABILITY) 2024-04-25T11:42:53.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:53 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2378177056' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T11:42:54.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:54 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1824677491' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T11:42:54.190 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299433 got 223338299438 for osd.7 2024-04-25T11:42:54.190 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.506+0000 7fe4dd925700 1 -- 172.21.15.83:0/2959425463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d8101030 msgr2=0x7fe4d8101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.506+0000 7fe4dd925700 1 --2- 172.21.15.83:0/2959425463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d8101030 0x7fe4d8101490 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009990 tx=0x7fe4cc009ca0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.506+0000 7fe4dd925700 1 -- 172.21.15.83:0/2959425463 shutdown_connections 2024-04-25T11:42:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.506+0000 7fe4dd925700 1 --2- 172.21.15.83:0/2959425463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d81019d0 0x7fe4d810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.506+0000 7fe4dd925700 1 --2- 172.21.15.83:0/2959425463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4d8108ad0 0x7fe4d8108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.506+0000 7fe4dd925700 1 --2- 172.21.15.83:0/2959425463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d8101030 0x7fe4d8101490 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.506+0000 7fe4dd925700 1 -- 172.21.15.83:0/2959425463 >> 172.21.15.83:0/2959425463 conn(0x7fe4d8077fb0 msgr2=0x7fe4d80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:54.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.506+0000 7fe4dd925700 1 -- 172.21.15.83:0/2959425463 shutdown_connections 2024-04-25T11:42:54.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.506+0000 7fe4dd925700 1 -- 172.21.15.83:0/2959425463 wait complete. 2024-04-25T11:42:54.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.507+0000 7fe4dd925700 1 Processor -- start 2024-04-25T11:42:54.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.507+0000 7fe4dd925700 1 -- start start 2024-04-25T11:42:54.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.507+0000 7fe4dd925700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d8101030 0x7fe4d81990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.507+0000 7fe4dd925700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4d81019d0 0x7fe4d8199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.507+0000 7fe4dd925700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d8108ad0 0x7fe4d819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.507+0000 7fe4dd925700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8112be0 con 0x7fe4d8101030 2024-04-25T11:42:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.507+0000 7fe4dd925700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8112a60 con 0x7fe4d8108ad0 2024-04-25T11:42:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.507+0000 7fe4dd925700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8112d60 con 0x7fe4d81019d0 2024-04-25T11:42:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.507+0000 7fe4d6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d8101030 0x7fe4d81990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.508+0000 7fe4d6ffd700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d8101030 0x7fe4d81990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41560/0 (socket says 172.21.15.83:41560) 2024-04-25T11:42:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.508+0000 7fe4d6ffd700 1 -- 172.21.15.83:0/843134787 learned_addr learned my addr 172.21.15.83:0/843134787 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.508+0000 7fe4d77fe700 1 --2- 172.21.15.83:0/843134787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d8108ad0 0x7fe4d819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.508+0000 7fe4d67fc700 1 --2- 172.21.15.83:0/843134787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4d81019d0 0x7fe4d8199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.508+0000 7fe4d6ffd700 1 -- 172.21.15.83:0/843134787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4d81019d0 msgr2=0x7fe4d8199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.508+0000 7fe4d6ffd700 1 --2- 172.21.15.83:0/843134787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4d81019d0 0x7fe4d8199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.508+0000 7fe4d6ffd700 1 -- 172.21.15.83:0/843134787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d8108ad0 msgr2=0x7fe4d819d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.508+0000 7fe4d6ffd700 1 --2- 172.21.15.83:0/843134787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d8108ad0 0x7fe4d819d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.508+0000 7fe4d6ffd700 1 -- 172.21.15.83:0/843134787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d819e0e0 con 0x7fe4d8101030 2024-04-25T11:42:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.508+0000 7fe4d6ffd700 1 --2- 172.21.15.83:0/843134787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d8101030 0x7fe4d81990d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800d980 tx=0x7fe4c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.508+0000 7fe4dc923700 1 -- 172.21.15.83:0/843134787 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8016070 con 0x7fe4d8101030 2024-04-25T11:42:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.508+0000 7fe4dc923700 1 -- 172.21.15.83:0/843134787 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4c8004830 con 0x7fe4d8101030 2024-04-25T11:42:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.508+0000 7fe4dd925700 1 -- 172.21.15.83:0/843134787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d819e3d0 con 0x7fe4d8101030 2024-04-25T11:42:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.508+0000 7fe4dc923700 1 -- 172.21.15.83:0/843134787 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8005540 con 0x7fe4d8101030 2024-04-25T11:42:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.508+0000 7fe4dd925700 1 -- 172.21.15.83:0/843134787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d81a5c50 con 0x7fe4d8101030 2024-04-25T11:42:54.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.509+0000 7fe4dd925700 1 -- 172.21.15.83:0/843134787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d8104350 con 0x7fe4d8101030 2024-04-25T11:42:54.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.511+0000 7fe4dc923700 1 -- 172.21.15.83:0/843134787 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe4c8004220 con 0x7fe4d8101030 2024-04-25T11:42:54.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.511+0000 7fe4dc923700 1 --2- 172.21.15.83:0/843134787 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe4c00a3cd0 0x7fe4c00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:54.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.511+0000 7fe4dc923700 1 -- 172.21.15.83:0/843134787 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fe4c80d07d0 con 0x7fe4d8101030 2024-04-25T11:42:54.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.511+0000 7fe4d67fc700 1 --2- 172.21.15.83:0/843134787 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe4c00a3cd0 0x7fe4c00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:54.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.512+0000 7fe4d67fc700 1 --2- 172.21.15.83:0/843134787 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe4c00a3cd0 0x7fe4c00a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0098e0 tx=0x7fe4cc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:54.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.515+0000 7fe4dc923700 1 -- 172.21.15.83:0/843134787 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4c801f020 con 0x7fe4d8101030 2024-04-25T11:42:54.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:54 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1824677491' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T11:42:54.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:54 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1824677491' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T11:42:54.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.720+0000 7fe4dd925700 1 -- 172.21.15.83:0/843134787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe4d819a460 con 0x7fe4d8101030 2024-04-25T11:42:54.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.720+0000 7fe4dc923700 1 -- 172.21.15.83:0/843134787 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe4d819a460 con 0x7fe4d8101030 2024-04-25T11:42:54.706 INFO:teuthology.orchestra.run.smithi083.stdout:77309411398 2024-04-25T11:42:54.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.722+0000 7fe4dd925700 1 -- 172.21.15.83:0/843134787 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe4c00a3cd0 msgr2=0x7fe4c00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.722+0000 7fe4dd925700 1 --2- 172.21.15.83:0/843134787 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe4c00a3cd0 0x7fe4c00a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0098e0 tx=0x7fe4cc005960 comp rx=0 tx=0).stop 2024-04-25T11:42:54.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.722+0000 7fe4dd925700 1 -- 172.21.15.83:0/843134787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d8101030 msgr2=0x7fe4d81990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:54.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.722+0000 7fe4dd925700 1 --2- 172.21.15.83:0/843134787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d8101030 0x7fe4d81990d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800d980 tx=0x7fe4c800dc90 comp rx=0 tx=0).stop 2024-04-25T11:42:54.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.722+0000 7fe4dd925700 1 -- 172.21.15.83:0/843134787 shutdown_connections 2024-04-25T11:42:54.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.722+0000 7fe4dd925700 1 --2- 172.21.15.83:0/843134787 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe4c00a3cd0 0x7fe4c00a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.722+0000 7fe4dd925700 1 --2- 172.21.15.83:0/843134787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4d8108ad0 0x7fe4d819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.722+0000 7fe4dd925700 1 --2- 172.21.15.83:0/843134787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d8101030 0x7fe4d81990d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.722+0000 7fe4dd925700 1 --2- 172.21.15.83:0/843134787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4d81019d0 0x7fe4d8199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:54.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.722+0000 7fe4dd925700 1 -- 172.21.15.83:0/843134787 >> 172.21.15.83:0/843134787 conn(0x7fe4d8077fb0 msgr2=0x7fe4d80ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:54.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.722+0000 7fe4dd925700 1 -- 172.21.15.83:0/843134787 shutdown_connections 2024-04-25T11:42:54.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:54.722+0000 7fe4dd925700 1 -- 172.21.15.83:0/843134787 wait complete. 2024-04-25T11:42:55.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:55 smithi083 ceph-mon[29330]: pgmap v320: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:42:55.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:55 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/843134787' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T11:42:55.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:55 smithi094 ceph-mon[33032]: pgmap v320: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:42:55.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:55 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/843134787' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T11:42:55.537 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411398 for osd.2 2024-04-25T11:42:55.537 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:55.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:55 smithi111 ceph-mon[32784]: pgmap v320: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:42:55.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:55 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/843134787' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T11:42:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.031+0000 7f29c3816700 1 -- 172.21.15.83:0/2287851421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29bc103720 msgr2=0x7f29bc10f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.031+0000 7f29c3816700 1 --2- 172.21.15.83:0/2287851421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29bc103720 0x7f29bc10f990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f29b400b0a0 tx=0x7f29b400b3b0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.031+0000 7f29c3816700 1 -- 172.21.15.83:0/2287851421 shutdown_connections 2024-04-25T11:42:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.031+0000 7f29c3816700 1 --2- 172.21.15.83:0/2287851421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29bc103720 0x7f29bc10f990 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.031+0000 7f29c3816700 1 --2- 172.21.15.83:0/2287851421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29bc108bc0 0x7f29bc108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.031+0000 7f29c3816700 1 --2- 172.21.15.83:0/2287851421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29bc102d60 0x7f29bc1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.031+0000 7f29c3816700 1 -- 172.21.15.83:0/2287851421 >> 172.21.15.83:0/2287851421 conn(0x7f29bc07a9d0 msgr2=0x7f29bc07ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.032+0000 7f29c3816700 1 -- 172.21.15.83:0/2287851421 shutdown_connections 2024-04-25T11:42:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.032+0000 7f29c3816700 1 -- 172.21.15.83:0/2287851421 wait complete. 2024-04-25T11:42:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.032+0000 7f29c3816700 1 Processor -- start 2024-04-25T11:42:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.032+0000 7f29c3816700 1 -- start start 2024-04-25T11:42:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.032+0000 7f29c3816700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29bc102d60 0x7f29bc199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.032+0000 7f29c3816700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29bc103720 0x7f29bc1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.032+0000 7f29c3816700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29bc108bc0 0x7f29bc19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.032+0000 7f29c3816700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc112d20 con 0x7f29bc108bc0 2024-04-25T11:42:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.032+0000 7f29c3816700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc112ba0 con 0x7f29bc103720 2024-04-25T11:42:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.032+0000 7f29c3816700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc112ea0 con 0x7f29bc102d60 2024-04-25T11:42:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.032+0000 7f29c1db3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29bc108bc0 0x7f29bc19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.032+0000 7f29c1db3700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29bc108bc0 0x7f29bc19da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41582/0 (socket says 172.21.15.83:41582) 2024-04-25T11:42:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.032+0000 7f29c1db3700 1 -- 172.21.15.83:0/1214268170 learned_addr learned my addr 172.21.15.83:0/1214268170 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.033+0000 7f29c15b2700 1 --2- 172.21.15.83:0/1214268170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29bc102d60 0x7f29bc199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.033+0000 7f29c0db1700 1 --2- 172.21.15.83:0/1214268170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29bc103720 0x7f29bc1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.033+0000 7f29c1db3700 1 -- 172.21.15.83:0/1214268170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29bc102d60 msgr2=0x7f29bc199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.033+0000 7f29c1db3700 1 --2- 172.21.15.83:0/1214268170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29bc102d60 0x7f29bc199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.033+0000 7f29c1db3700 1 -- 172.21.15.83:0/1214268170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29bc103720 msgr2=0x7f29bc1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.033+0000 7f29c1db3700 1 --2- 172.21.15.83:0/1214268170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29bc103720 0x7f29bc1996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.033+0000 7f29c1db3700 1 -- 172.21.15.83:0/1214268170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc19e180 con 0x7f29bc108bc0 2024-04-25T11:42:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.033+0000 7f29c0db1700 1 --2- 172.21.15.83:0/1214268170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29bc103720 0x7f29bc1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:42:56.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.033+0000 7f29c1db3700 1 --2- 172.21.15.83:0/1214268170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29bc108bc0 0x7f29bc19da40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f29b400b830 tx=0x7f29b4004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:56.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.033+0000 7f29b27fc700 1 -- 172.21.15.83:0/1214268170 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4021070 con 0x7f29bc108bc0 2024-04-25T11:42:56.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.033+0000 7f29b27fc700 1 -- 172.21.15.83:0/1214268170 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29b4007e30 con 0x7f29bc108bc0 2024-04-25T11:42:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.033+0000 7f29c3816700 1 -- 172.21.15.83:0/1214268170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29bc19e410 con 0x7f29bc108bc0 2024-04-25T11:42:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.033+0000 7f29b27fc700 1 -- 172.21.15.83:0/1214268170 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4004660 con 0x7f29bc108bc0 2024-04-25T11:42:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.033+0000 7f29c3816700 1 -- 172.21.15.83:0/1214268170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29bc1a5cf0 con 0x7f29bc108bc0 2024-04-25T11:42:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.034+0000 7f29c3816700 1 -- 172.21.15.83:0/1214268170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29bc069730 con 0x7f29bc108bc0 2024-04-25T11:42:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.035+0000 7f29b27fc700 1 -- 172.21.15.83:0/1214268170 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f29b4003c00 con 0x7f29bc108bc0 2024-04-25T11:42:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.036+0000 7f29b27fc700 1 --2- 172.21.15.83:0/1214268170 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f29a80a3cd0 0x7f29a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.036+0000 7f29b27fc700 1 -- 172.21.15.83:0/1214268170 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f29b400f020 con 0x7f29bc108bc0 2024-04-25T11:42:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.036+0000 7f29c15b2700 1 --2- 172.21.15.83:0/1214268170 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f29a80a3cd0 0x7f29a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.037+0000 7f29c15b2700 1 --2- 172.21.15.83:0/1214268170 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f29a80a3cd0 0x7f29a80a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f29b8005950 tx=0x7f29b80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:56.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.040+0000 7f29b27fc700 1 -- 172.21.15.83:0/1214268170 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29b402a020 con 0x7f29bc108bc0 2024-04-25T11:42:56.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.259+0000 7f29c3816700 1 -- 172.21.15.83:0/1214268170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f29bc19a4d0 con 0x7f29bc108bc0 2024-04-25T11:42:56.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.260+0000 7f29b27fc700 1 -- 172.21.15.83:0/1214268170 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f29b409e920 con 0x7f29bc108bc0 2024-04-25T11:42:56.246 INFO:teuthology.orchestra.run.smithi083.stdout:257698037801 2024-04-25T11:42:56.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.262+0000 7f29c3816700 1 -- 172.21.15.83:0/1214268170 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f29a80a3cd0 msgr2=0x7f29a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.262+0000 7f29c3816700 1 --2- 172.21.15.83:0/1214268170 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f29a80a3cd0 0x7f29a80a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f29b8005950 tx=0x7f29b80058e0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.262+0000 7f29c3816700 1 -- 172.21.15.83:0/1214268170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29bc108bc0 msgr2=0x7f29bc19da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:56.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.262+0000 7f29c3816700 1 --2- 172.21.15.83:0/1214268170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29bc108bc0 0x7f29bc19da40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f29b400b830 tx=0x7f29b4004d90 comp rx=0 tx=0).stop 2024-04-25T11:42:56.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.262+0000 7f29c3816700 1 -- 172.21.15.83:0/1214268170 shutdown_connections 2024-04-25T11:42:56.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.262+0000 7f29c3816700 1 --2- 172.21.15.83:0/1214268170 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f29a80a3cd0 0x7f29a80a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.262+0000 7f29c3816700 1 --2- 172.21.15.83:0/1214268170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29bc108bc0 0x7f29bc19da40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.262+0000 7f29c3816700 1 --2- 172.21.15.83:0/1214268170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29bc102d60 0x7f29bc199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.262+0000 7f29c3816700 1 --2- 172.21.15.83:0/1214268170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29bc103720 0x7f29bc1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:56.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.262+0000 7f29c3816700 1 -- 172.21.15.83:0/1214268170 >> 172.21.15.83:0/1214268170 conn(0x7f29bc07a9d0 msgr2=0x7f29bc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:56.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.262+0000 7f29c3816700 1 -- 172.21.15.83:0/1214268170 shutdown_connections 2024-04-25T11:42:56.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:56.262+0000 7f29c3816700 1 -- 172.21.15.83:0/1214268170 wait complete. 2024-04-25T11:42:56.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:56 smithi083 ceph-mon[29330]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T11:42:56.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:56 smithi083 ceph-mon[29330]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-04-25T11:42:56.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:56 smithi083 ceph-mon[29330]: Cluster is now healthy 2024-04-25T11:42:56.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:56 smithi094 ceph-mon[33032]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T11:42:56.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:56 smithi094 ceph-mon[33032]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-04-25T11:42:56.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:56 smithi094 ceph-mon[33032]: Cluster is now healthy 2024-04-25T11:42:56.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:56 smithi111 ceph-mon[32784]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T11:42:56.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:56 smithi111 ceph-mon[32784]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-04-25T11:42:56.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:56 smithi111 ceph-mon[32784]: Cluster is now healthy 2024-04-25T11:42:56.945 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037795 got 257698037801 for osd.8 2024-04-25T11:42:56.945 DEBUG:teuthology.parallel:result is None 2024-04-25T11:42:56.946 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T11:42:56.946 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T11:42:57.193 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:42:57.515 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:57 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:57.515 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:57 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:57.515 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:57 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1214268170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T11:42:57.515 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:57 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:42:57.515 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:57 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:42:57.515 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:57 smithi083 ceph-mon[29330]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:57.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:57 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:57.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:57 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:57.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:57 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1214268170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T11:42:57.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:57 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:42:57.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:57 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:42:57.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:57 smithi094 ceph-mon[33032]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:57.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:57 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:57.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:57 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:57.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:57 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1214268170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T11:42:57.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:57 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:42:57.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:57 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:42:57.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:57 smithi111 ceph-mon[32784]: from='mgr.14152 172.21.15.83:0/3772543286' entity='mgr.x' 2024-04-25T11:42:58.302 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:58 smithi083 ceph-mon[29330]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T11:42:58.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:58 smithi094 ceph-mon[33032]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T11:42:58.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:58 smithi111 ceph-mon[32784]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T11:42:58.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.854+0000 7f7c88778700 1 -- 172.21.15.83:0/694456883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c80101940 msgr2=0x7f7c80101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:58.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.854+0000 7f7c88778700 1 --2- 172.21.15.83:0/694456883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c80101940 0x7f7c80101dc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009990 tx=0x7f7c7c009ca0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.854+0000 7f7c88778700 1 -- 172.21.15.83:0/694456883 shutdown_connections 2024-04-25T11:42:58.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.854+0000 7f7c88778700 1 --2- 172.21.15.83:0/694456883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c80102300 0x7f7c8010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.854+0000 7f7c88778700 1 --2- 172.21.15.83:0/694456883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c80107990 0x7f7c80107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.854+0000 7f7c88778700 1 --2- 172.21.15.83:0/694456883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c80101940 0x7f7c80101dc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.854+0000 7f7c88778700 1 -- 172.21.15.83:0/694456883 >> 172.21.15.83:0/694456883 conn(0x7f7c800788f0 msgr2=0x7f7c80078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:58.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.854+0000 7f7c88778700 1 -- 172.21.15.83:0/694456883 shutdown_connections 2024-04-25T11:42:58.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.854+0000 7f7c88778700 1 -- 172.21.15.83:0/694456883 wait complete. 2024-04-25T11:42:58.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c88778700 1 Processor -- start 2024-04-25T11:42:58.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c88778700 1 -- start start 2024-04-25T11:42:58.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c88778700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c80101940 0x7f7c80198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c88778700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c80102300 0x7f7c801986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c88778700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c80107990 0x7f7c8019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c88778700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7c80111bd0 con 0x7f7c80102300 2024-04-25T11:42:58.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c88778700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7c80111a50 con 0x7f7c80101940 2024-04-25T11:42:58.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c88778700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7c80111d50 con 0x7f7c80107990 2024-04-25T11:42:58.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c85d13700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c80102300 0x7f7c801986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c85d13700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c80102300 0x7f7c801986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41592/0 (socket says 172.21.15.83:41592) 2024-04-25T11:42:58.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c85d13700 1 -- 172.21.15.83:0/587809819 learned_addr learned my addr 172.21.15.83:0/587809819 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:42:58.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c86514700 1 --2- 172.21.15.83:0/587809819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c80101940 0x7f7c80198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c85d13700 1 -- 172.21.15.83:0/587809819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c80107990 msgr2=0x7f7c8019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:58.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c85d13700 1 --2- 172.21.15.83:0/587809819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c80107990 0x7f7c8019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c85d13700 1 -- 172.21.15.83:0/587809819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c80101940 msgr2=0x7f7c80198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:58.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c85d13700 1 --2- 172.21.15.83:0/587809819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c80101940 0x7f7c80198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:58.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c85d13700 1 -- 172.21.15.83:0/587809819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8019d170 con 0x7f7c80102300 2024-04-25T11:42:58.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.855+0000 7f7c85d13700 1 --2- 172.21.15.83:0/587809819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c80102300 0x7f7c801986a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c000c00 tx=0x7f7c7c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:58.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.856+0000 7f7c777fe700 1 -- 172.21.15.83:0/587809819 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c02a070 con 0x7f7c80102300 2024-04-25T11:42:58.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.856+0000 7f7c777fe700 1 -- 172.21.15.83:0/587809819 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c7c01ad70 con 0x7f7c80102300 2024-04-25T11:42:58.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.856+0000 7f7c88778700 1 -- 172.21.15.83:0/587809819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8019d400 con 0x7f7c80102300 2024-04-25T11:42:58.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.856+0000 7f7c777fe700 1 -- 172.21.15.83:0/587809819 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c00f460 con 0x7f7c80102300 2024-04-25T11:42:58.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.856+0000 7f7c88778700 1 -- 172.21.15.83:0/587809819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c8006a720 con 0x7f7c80102300 2024-04-25T11:42:58.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.856+0000 7f7c88778700 1 -- 172.21.15.83:0/587809819 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c80103060 con 0x7f7c80102300 2024-04-25T11:42:58.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.858+0000 7f7c777fe700 1 -- 172.21.15.83:0/587809819 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7c7c02d070 con 0x7f7c80102300 2024-04-25T11:42:58.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.858+0000 7f7c777fe700 1 --2- 172.21.15.83:0/587809819 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c6c0a3cd0 0x7f7c6c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:42:58.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.858+0000 7f7c86514700 1 --2- 172.21.15.83:0/587809819 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c6c0a3cd0 0x7f7c6c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:42:58.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.858+0000 7f7c777fe700 1 -- 172.21.15.83:0/587809819 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f7c7c0d3a60 con 0x7f7c80102300 2024-04-25T11:42:58.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.859+0000 7f7c86514700 1 --2- 172.21.15.83:0/587809819 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c6c0a3cd0 0x7f7c6c0a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c70006fd0 tx=0x7f7c70008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:42:58.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:58.862+0000 7f7c777fe700 1 -- 172.21.15.83:0/587809819 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c7c034070 con 0x7f7c80102300 2024-04-25T11:42:59.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:59.056+0000 7f7c88778700 1 -- 172.21.15.83:0/587809819 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c8006aa40 con 0x7f7c6c0a3cd0 2024-04-25T11:42:59.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:59.063+0000 7f7c777fe700 1 -- 172.21.15.83:0/587809819 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258743 (secure 0 0 0) 0x7f7c8006aa40 con 0x7f7c6c0a3cd0 2024-04-25T11:42:59.053 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:42:59.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:59.066+0000 7f7c88778700 1 -- 172.21.15.83:0/587809819 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c6c0a3cd0 msgr2=0x7f7c6c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:59.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:59.066+0000 7f7c88778700 1 --2- 172.21.15.83:0/587809819 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c6c0a3cd0 0x7f7c6c0a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7c70006fd0 tx=0x7f7c70008040 comp rx=0 tx=0).stop 2024-04-25T11:42:59.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:59.066+0000 7f7c88778700 1 -- 172.21.15.83:0/587809819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c80102300 msgr2=0x7f7c801986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:42:59.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:59.066+0000 7f7c88778700 1 --2- 172.21.15.83:0/587809819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c80102300 0x7f7c801986a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c000c00 tx=0x7f7c7c0047b0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:59.066+0000 7f7c88778700 1 -- 172.21.15.83:0/587809819 shutdown_connections 2024-04-25T11:42:59.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:59.066+0000 7f7c88778700 1 --2- 172.21.15.83:0/587809819 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7c6c0a3cd0 0x7f7c6c0a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:59.066+0000 7f7c88778700 1 --2- 172.21.15.83:0/587809819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c80107990 0x7f7c8019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:59.066+0000 7f7c88778700 1 --2- 172.21.15.83:0/587809819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c80101940 0x7f7c80198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:59.066+0000 7f7c88778700 1 --2- 172.21.15.83:0/587809819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7c80102300 0x7f7c801986a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:42:59.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:59.066+0000 7f7c88778700 1 -- 172.21.15.83:0/587809819 >> 172.21.15.83:0/587809819 conn(0x7f7c800788f0 msgr2=0x7f7c800ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:42:59.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:59.066+0000 7f7c88778700 1 -- 172.21.15.83:0/587809819 shutdown_connections 2024-04-25T11:42:59.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:42:59.066+0000 7f7c88778700 1 -- 172.21.15.83:0/587809819 wait complete. 2024-04-25T11:42:59.055 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T11:42:59.376 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:42:59 smithi083 ceph-mon[29330]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:42:59.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:42:59 smithi094 ceph-mon[33032]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:42:59.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:42:59 smithi111 ceph-mon[32784]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:42:59.687 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-04-25T11:42:57.926328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656718,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":805,"num_read_kb":524,"num_write":417,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":295,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375116,"kb_used_data":13404,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566708,"statfs":{"total":1151940427776,"available":1151556308992,"internally_reserved":0,"allocated":13725696,"data_stored":7480998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":63,"num_read_kb":28,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002401"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078580+0000","last_change":"2024-04-25T11:41:44.907326+0000","last_active":"2024-04-25T11:42:51.078580+0000","last_peered":"2024-04-25T11:42:51.078580+0000","last_clean":"2024-04-25T11:42:51.078580+0000","last_became_active":"2024-04-25T11:41:44.906964+0000","last_became_peered":"2024-04-25T11:41:44.906964+0000","last_unstale":"2024-04-25T11:42:51.078580+0000","last_undegraded":"2024-04-25T11:42:51.078580+0000","last_fullsized":"2024-04-25T11:42:51.078580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:46.656252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080527+0000","last_change":"2024-04-25T11:41:44.907601+0000","last_active":"2024-04-25T11:42:51.080527+0000","last_peered":"2024-04-25T11:42:51.080527+0000","last_clean":"2024-04-25T11:42:51.080527+0000","last_became_active":"2024-04-25T11:41:44.907142+0000","last_became_peered":"2024-04-25T11:41:44.907142+0000","last_unstale":"2024-04-25T11:42:51.080527+0000","last_undegraded":"2024-04-25T11:42:51.080527+0000","last_fullsized":"2024-04-25T11:42:51.080527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:41.901259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080315+0000","last_change":"2024-04-25T11:41:44.910479+0000","last_active":"2024-04-25T11:42:51.080315+0000","last_peered":"2024-04-25T11:42:51.080315+0000","last_clean":"2024-04-25T11:42:51.080315+0000","last_became_active":"2024-04-25T11:41:44.909711+0000","last_became_peered":"2024-04-25T11:41:44.909711+0000","last_unstale":"2024-04-25T11:42:51.080315+0000","last_undegraded":"2024-04-25T11:42:51.080315+0000","last_fullsized":"2024-04-25T11:42:51.080315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.586793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.599617+0000","last_change":"2024-04-25T11:41:44.909936+0000","last_active":"2024-04-25T11:42:51.599617+0000","last_peered":"2024-04-25T11:42:51.599617+0000","last_clean":"2024-04-25T11:42:51.599617+0000","last_became_active":"2024-04-25T11:41:44.909645+0000","last_became_peered":"2024-04-25T11:41:44.909645+0000","last_unstale":"2024-04-25T11:42:51.599617+0000","last_undegraded":"2024-04-25T11:42:51.599617+0000","last_fullsized":"2024-04-25T11:42:51.599617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:12.518017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080175+0000","last_change":"2024-04-25T11:41:44.913907+0000","last_active":"2024-04-25T11:42:51.080175+0000","last_peered":"2024-04-25T11:42:51.080175+0000","last_clean":"2024-04-25T11:42:51.080175+0000","last_became_active":"2024-04-25T11:41:44.912593+0000","last_became_peered":"2024-04-25T11:41:44.912593+0000","last_unstale":"2024-04-25T11:42:51.080175+0000","last_undegraded":"2024-04-25T11:42:51.080175+0000","last_fullsized":"2024-04-25T11:42:51.080175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:40.164688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078266+0000","last_change":"2024-04-25T11:41:44.903237+0000","last_active":"2024-04-25T11:42:51.078266+0000","last_peered":"2024-04-25T11:42:51.078266+0000","last_clean":"2024-04-25T11:42:51.078266+0000","last_became_active":"2024-04-25T11:41:44.902881+0000","last_became_peered":"2024-04-25T11:41:44.902881+0000","last_unstale":"2024-04-25T11:42:51.078266+0000","last_undegraded":"2024-04-25T11:42:51.078266+0000","last_fullsized":"2024-04-25T11:42:51.078266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:36.201322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080392+0000","last_change":"2024-04-25T11:41:44.907497+0000","last_active":"2024-04-25T11:42:51.080392+0000","last_peered":"2024-04-25T11:42:51.080392+0000","last_clean":"2024-04-25T11:42:51.080392+0000","last_became_active":"2024-04-25T11:41:44.907117+0000","last_became_peered":"2024-04-25T11:41:44.907117+0000","last_unstale":"2024-04-25T11:42:51.080392+0000","last_undegraded":"2024-04-25T11:42:51.080392+0000","last_fullsized":"2024-04-25T11:42:51.080392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:25.987238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.597914+0000","last_change":"2024-04-25T11:41:44.915499+0000","last_active":"2024-04-25T11:42:51.597914+0000","last_peered":"2024-04-25T11:42:51.597914+0000","last_clean":"2024-04-25T11:42:51.597914+0000","last_became_active":"2024-04-25T11:41:44.915221+0000","last_became_peered":"2024-04-25T11:41:44.915221+0000","last_unstale":"2024-04-25T11:42:51.597914+0000","last_undegraded":"2024-04-25T11:42:51.597914+0000","last_fullsized":"2024-04-25T11:42:51.597914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:36.717428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598840+0000","last_change":"2024-04-25T11:41:44.910599+0000","last_active":"2024-04-25T11:42:51.598840+0000","last_peered":"2024-04-25T11:42:51.598840+0000","last_clean":"2024-04-25T11:42:51.598840+0000","last_became_active":"2024-04-25T11:41:44.908747+0000","last_became_peered":"2024-04-25T11:41:44.908747+0000","last_unstale":"2024-04-25T11:42:51.598840+0000","last_undegraded":"2024-04-25T11:42:51.598840+0000","last_fullsized":"2024-04-25T11:42:51.598840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:50.281168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078658+0000","last_change":"2024-04-25T11:41:44.910692+0000","last_active":"2024-04-25T11:42:51.078658+0000","last_peered":"2024-04-25T11:42:51.078658+0000","last_clean":"2024-04-25T11:42:51.078658+0000","last_became_active":"2024-04-25T11:41:44.910343+0000","last_became_peered":"2024-04-25T11:41:44.910343+0000","last_unstale":"2024-04-25T11:42:51.078658+0000","last_undegraded":"2024-04-25T11:42:51.078658+0000","last_fullsized":"2024-04-25T11:42:51.078658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.945563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080980+0000","last_change":"2024-04-25T11:41:44.910751+0000","last_active":"2024-04-25T11:42:51.080980+0000","last_peered":"2024-04-25T11:42:51.080980+0000","last_clean":"2024-04-25T11:42:51.080980+0000","last_became_active":"2024-04-25T11:41:44.909686+0000","last_became_peered":"2024-04-25T11:41:44.909686+0000","last_unstale":"2024-04-25T11:42:51.080980+0000","last_undegraded":"2024-04-25T11:42:51.080980+0000","last_fullsized":"2024-04-25T11:42:51.080980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:37.216282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.599397+0000","last_change":"2024-04-25T11:41:44.915148+0000","last_active":"2024-04-25T11:42:51.599397+0000","last_peered":"2024-04-25T11:42:51.599397+0000","last_clean":"2024-04-25T11:42:51.599397+0000","last_became_active":"2024-04-25T11:41:44.914772+0000","last_became_peered":"2024-04-25T11:41:44.914772+0000","last_unstale":"2024-04-25T11:42:51.599397+0000","last_undegraded":"2024-04-25T11:42:51.599397+0000","last_fullsized":"2024-04-25T11:42:51.599397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:09.121386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.579956+0000","last_change":"2024-04-25T11:41:44.906569+0000","last_active":"2024-04-25T11:42:51.579956+0000","last_peered":"2024-04-25T11:42:51.579956+0000","last_clean":"2024-04-25T11:42:51.579956+0000","last_became_active":"2024-04-25T11:41:44.906259+0000","last_became_peered":"2024-04-25T11:41:44.906259+0000","last_unstale":"2024-04-25T11:42:51.579956+0000","last_undegraded":"2024-04-25T11:42:51.579956+0000","last_fullsized":"2024-04-25T11:42:51.579956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:47.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.079880+0000","last_change":"2024-04-25T11:41:44.909261+0000","last_active":"2024-04-25T11:42:51.079880+0000","last_peered":"2024-04-25T11:42:51.079880+0000","last_clean":"2024-04-25T11:42:51.079880+0000","last_became_active":"2024-04-25T11:41:44.907931+0000","last_became_peered":"2024-04-25T11:41:44.907931+0000","last_unstale":"2024-04-25T11:42:51.079880+0000","last_undegraded":"2024-04-25T11:42:51.079880+0000","last_fullsized":"2024-04-25T11:42:51.079880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:35.754990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.079710+0000","last_change":"2024-04-25T11:41:44.908695+0000","last_active":"2024-04-25T11:42:51.079710+0000","last_peered":"2024-04-25T11:42:51.079710+0000","last_clean":"2024-04-25T11:42:51.079710+0000","last_became_active":"2024-04-25T11:41:44.908374+0000","last_became_peered":"2024-04-25T11:41:44.908374+0000","last_unstale":"2024-04-25T11:42:51.079710+0000","last_undegraded":"2024-04-25T11:42:51.079710+0000","last_fullsized":"2024-04-25T11:42:51.079710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:30.120631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598258+0000","last_change":"2024-04-25T11:41:44.903279+0000","last_active":"2024-04-25T11:42:51.598258+0000","last_peered":"2024-04-25T11:42:51.598258+0000","last_clean":"2024-04-25T11:42:51.598258+0000","last_became_active":"2024-04-25T11:41:44.902958+0000","last_became_peered":"2024-04-25T11:41:44.902958+0000","last_unstale":"2024-04-25T11:42:51.598258+0000","last_undegraded":"2024-04-25T11:42:51.598258+0000","last_fullsized":"2024-04-25T11:42:51.598258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:18.380774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078846+0000","last_change":"2024-04-25T11:41:44.908108+0000","last_active":"2024-04-25T11:42:51.078846+0000","last_peered":"2024-04-25T11:42:51.078846+0000","last_clean":"2024-04-25T11:42:51.078846+0000","last_became_active":"2024-04-25T11:41:44.907787+0000","last_became_peered":"2024-04-25T11:41:44.907787+0000","last_unstale":"2024-04-25T11:42:51.078846+0000","last_undegraded":"2024-04-25T11:42:51.078846+0000","last_fullsized":"2024-04-25T11:42:51.078846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:53.306845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078226+0000","last_change":"2024-04-25T11:41:44.906630+0000","last_active":"2024-04-25T11:42:51.078226+0000","last_peered":"2024-04-25T11:42:51.078226+0000","last_clean":"2024-04-25T11:42:51.078226+0000","last_became_active":"2024-04-25T11:41:44.906321+0000","last_became_peered":"2024-04-25T11:41:44.906321+0000","last_unstale":"2024-04-25T11:42:51.078226+0000","last_undegraded":"2024-04-25T11:42:51.078226+0000","last_fullsized":"2024-04-25T11:42:51.078226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:25.588918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081862+0000","last_change":"2024-04-25T11:41:44.905390+0000","last_active":"2024-04-25T11:42:51.081862+0000","last_peered":"2024-04-25T11:42:51.081862+0000","last_clean":"2024-04-25T11:42:51.081862+0000","last_became_active":"2024-04-25T11:41:44.905033+0000","last_became_peered":"2024-04-25T11:41:44.905033+0000","last_unstale":"2024-04-25T11:42:51.081862+0000","last_undegraded":"2024-04-25T11:42:51.081862+0000","last_fullsized":"2024-04-25T11:42:51.081862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:00.998434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:53.670832+0000","last_change":"2024-04-25T11:41:44.454718+0000","last_active":"2024-04-25T11:42:53.670832+0000","last_peered":"2024-04-25T11:42:53.670832+0000","last_clean":"2024-04-25T11:42:53.670832+0000","last_became_active":"2024-04-25T11:41:44.454400+0000","last_became_peered":"2024-04-25T11:41:44.454400+0000","last_unstale":"2024-04-25T11:42:53.670832+0000","last_undegraded":"2024-04-25T11:42:53.670832+0000","last_fullsized":"2024-04-25T11:42:53.670832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:05.965988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598363+0000","last_change":"2024-04-25T11:41:44.913512+0000","last_active":"2024-04-25T11:42:51.598363+0000","last_peered":"2024-04-25T11:42:51.598363+0000","last_clean":"2024-04-25T11:42:51.598363+0000","last_became_active":"2024-04-25T11:41:44.913168+0000","last_became_peered":"2024-04-25T11:41:44.913168+0000","last_unstale":"2024-04-25T11:42:51.598363+0000","last_undegraded":"2024-04-25T11:42:51.598363+0000","last_fullsized":"2024-04-25T11:42:51.598363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:25.139747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598473+0000","last_change":"2024-04-25T11:41:44.451331+0000","last_active":"2024-04-25T11:42:51.598473+0000","last_peered":"2024-04-25T11:42:51.598473+0000","last_clean":"2024-04-25T11:42:51.598473+0000","last_became_active":"2024-04-25T11:41:44.450788+0000","last_became_peered":"2024-04-25T11:41:44.450788+0000","last_unstale":"2024-04-25T11:42:51.598473+0000","last_undegraded":"2024-04-25T11:42:51.598473+0000","last_fullsized":"2024-04-25T11:42:51.598473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:49.011558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080471+0000","last_change":"2024-04-25T11:41:44.908536+0000","last_active":"2024-04-25T11:42:51.080471+0000","last_peered":"2024-04-25T11:42:51.080471+0000","last_clean":"2024-04-25T11:42:51.080471+0000","last_became_active":"2024-04-25T11:41:44.907823+0000","last_became_peered":"2024-04-25T11:41:44.907823+0000","last_unstale":"2024-04-25T11:42:51.080471+0000","last_undegraded":"2024-04-25T11:42:51.080471+0000","last_fullsized":"2024-04-25T11:42:51.080471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:19.898695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.095130+0000","last_change":"2024-04-25T11:42:51.094639+0000","last_active":"2024-04-25T11:42:51.095130+0000","last_peered":"2024-04-25T11:42:51.095130+0000","last_clean":"2024-04-25T11:42:51.095130+0000","last_became_active":"2024-04-25T11:42:51.083342+0000","last_became_peered":"2024-04-25T11:42:51.083342+0000","last_unstale":"2024-04-25T11:42:51.095130+0000","last_undegraded":"2024-04-25T11:42:51.095130+0000","last_fullsized":"2024-04-25T11:42:51.095130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:20.075324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598595+0000","last_change":"2024-04-25T11:41:44.908881+0000","last_active":"2024-04-25T11:42:51.598595+0000","last_peered":"2024-04-25T11:42:51.598595+0000","last_clean":"2024-04-25T11:42:51.598595+0000","last_became_active":"2024-04-25T11:41:44.908493+0000","last_became_peered":"2024-04-25T11:41:44.908493+0000","last_unstale":"2024-04-25T11:42:51.598595+0000","last_undegraded":"2024-04-25T11:42:51.598595+0000","last_fullsized":"2024-04-25T11:42:51.598595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:58.930420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080533+0000","last_change":"2024-04-25T11:41:44.451870+0000","last_active":"2024-04-25T11:42:51.080533+0000","last_peered":"2024-04-25T11:42:51.080533+0000","last_clean":"2024-04-25T11:42:51.080533+0000","last_became_active":"2024-04-25T11:41:44.451511+0000","last_became_peered":"2024-04-25T11:41:44.451511+0000","last_unstale":"2024-04-25T11:42:51.080533+0000","last_undegraded":"2024-04-25T11:42:51.080533+0000","last_fullsized":"2024-04-25T11:42:51.080533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:26.457919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.380169+0000","last_change":"2024-04-25T11:41:44.910738+0000","last_active":"2024-04-25T11:42:51.380169+0000","last_peered":"2024-04-25T11:42:51.380169+0000","last_clean":"2024-04-25T11:42:51.380169+0000","last_became_active":"2024-04-25T11:41:44.910415+0000","last_became_peered":"2024-04-25T11:41:44.910415+0000","last_unstale":"2024-04-25T11:42:51.380169+0000","last_undegraded":"2024-04-25T11:42:51.380169+0000","last_fullsized":"2024-04-25T11:42:51.380169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:01.333894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":90,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:52.664276+0000","last_change":"2024-04-25T11:41:44.455288+0000","last_active":"2024-04-25T11:42:52.664276+0000","last_peered":"2024-04-25T11:42:52.664276+0000","last_clean":"2024-04-25T11:42:52.664276+0000","last_became_active":"2024-04-25T11:41:44.454965+0000","last_became_peered":"2024-04-25T11:41:44.454965+0000","last_unstale":"2024-04-25T11:42:52.664276+0000","last_undegraded":"2024-04-25T11:42:52.664276+0000","last_fullsized":"2024-04-25T11:42:52.664276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:33.943287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.083170+0000","last_change":"2024-04-25T11:42:51.083170+0000","last_active":"2024-04-25T11:42:51.083170+0000","last_peered":"2024-04-25T11:42:51.083170+0000","last_clean":"2024-04-25T11:42:51.083170+0000","last_became_active":"2024-04-25T11:42:51.082750+0000","last_became_peered":"2024-04-25T11:42:51.082750+0000","last_unstale":"2024-04-25T11:42:51.083170+0000","last_undegraded":"2024-04-25T11:42:51.083170+0000","last_fullsized":"2024-04-25T11:42:51.083170+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:13.464879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078225+0000","last_change":"2024-04-25T11:41:44.451846+0000","last_active":"2024-04-25T11:42:51.078225+0000","last_peered":"2024-04-25T11:42:51.078225+0000","last_clean":"2024-04-25T11:42:51.078225+0000","last_became_active":"2024-04-25T11:41:44.451416+0000","last_became_peered":"2024-04-25T11:41:44.451416+0000","last_unstale":"2024-04-25T11:42:51.078225+0000","last_undegraded":"2024-04-25T11:42:51.078225+0000","last_fullsized":"2024-04-25T11:42:51.078225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:57.023883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.379919+0000","last_change":"2024-04-25T11:41:44.907761+0000","last_active":"2024-04-25T11:42:51.379919+0000","last_peered":"2024-04-25T11:42:51.379919+0000","last_clean":"2024-04-25T11:42:51.379919+0000","last_became_active":"2024-04-25T11:41:44.906743+0000","last_became_peered":"2024-04-25T11:41:44.906743+0000","last_unstale":"2024-04-25T11:42:51.379919+0000","last_undegraded":"2024-04-25T11:42:51.379919+0000","last_fullsized":"2024-04-25T11:42:51.379919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:43.304985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.579856+0000","last_change":"2024-04-25T11:41:44.447415+0000","last_active":"2024-04-25T11:42:51.579856+0000","last_peered":"2024-04-25T11:42:51.579856+0000","last_clean":"2024-04-25T11:42:51.579856+0000","last_became_active":"2024-04-25T11:41:44.447091+0000","last_became_peered":"2024-04-25T11:41:44.447091+0000","last_unstale":"2024-04-25T11:42:51.579856+0000","last_undegraded":"2024-04-25T11:42:51.579856+0000","last_fullsized":"2024-04-25T11:42:51.579856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:27.879115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598001+0000","last_change":"2024-04-25T11:41:44.913350+0000","last_active":"2024-04-25T11:42:51.598001+0000","last_peered":"2024-04-25T11:42:51.598001+0000","last_clean":"2024-04-25T11:42:51.598001+0000","last_became_active":"2024-04-25T11:41:44.912602+0000","last_became_peered":"2024-04-25T11:41:44.912602+0000","last_unstale":"2024-04-25T11:42:51.598001+0000","last_undegraded":"2024-04-25T11:42:51.598001+0000","last_fullsized":"2024-04-25T11:42:51.598001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:03.099520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.379589+0000","last_change":"2024-04-25T11:41:44.451283+0000","last_active":"2024-04-25T11:42:51.379589+0000","last_peered":"2024-04-25T11:42:51.379589+0000","last_clean":"2024-04-25T11:42:51.379589+0000","last_became_active":"2024-04-25T11:41:44.450955+0000","last_became_peered":"2024-04-25T11:41:44.450955+0000","last_unstale":"2024-04-25T11:42:51.379589+0000","last_undegraded":"2024-04-25T11:42:51.379589+0000","last_fullsized":"2024-04-25T11:42:51.379589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:00.912227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.077923+0000","last_change":"2024-04-25T11:41:44.904055+0000","last_active":"2024-04-25T11:42:51.077923+0000","last_peered":"2024-04-25T11:42:51.077923+0000","last_clean":"2024-04-25T11:42:51.077923+0000","last_became_active":"2024-04-25T11:41:44.903731+0000","last_became_peered":"2024-04-25T11:41:44.903731+0000","last_unstale":"2024-04-25T11:42:51.077923+0000","last_undegraded":"2024-04-25T11:42:51.077923+0000","last_fullsized":"2024-04-25T11:42:51.077923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:09.912305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":149,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:53.664953+0000","last_change":"2024-04-25T11:41:44.444989+0000","last_active":"2024-04-25T11:42:53.664953+0000","last_peered":"2024-04-25T11:42:53.664953+0000","last_clean":"2024-04-25T11:42:53.664953+0000","last_became_active":"2024-04-25T11:41:44.444543+0000","last_became_peered":"2024-04-25T11:41:44.444543+0000","last_unstale":"2024-04-25T11:42:53.664953+0000","last_undegraded":"2024-04-25T11:42:53.664953+0000","last_fullsized":"2024-04-25T11:42:53.664953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:08.957068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598644+0000","last_change":"2024-04-25T11:41:44.907435+0000","last_active":"2024-04-25T11:42:51.598644+0000","last_peered":"2024-04-25T11:42:51.598644+0000","last_clean":"2024-04-25T11:42:51.598644+0000","last_became_active":"2024-04-25T11:41:44.906972+0000","last_became_peered":"2024-04-25T11:41:44.906972+0000","last_unstale":"2024-04-25T11:42:51.598644+0000","last_undegraded":"2024-04-25T11:42:51.598644+0000","last_fullsized":"2024-04-25T11:42:51.598644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:47.011440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081068+0000","last_change":"2024-04-25T11:41:44.456804+0000","last_active":"2024-04-25T11:42:51.081068+0000","last_peered":"2024-04-25T11:42:51.081068+0000","last_clean":"2024-04-25T11:42:51.081068+0000","last_became_active":"2024-04-25T11:41:44.456472+0000","last_became_peered":"2024-04-25T11:41:44.456472+0000","last_unstale":"2024-04-25T11:42:51.081068+0000","last_undegraded":"2024-04-25T11:42:51.081068+0000","last_fullsized":"2024-04-25T11:42:51.081068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:49.293772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.581037+0000","last_change":"2024-04-25T11:42:51.581037+0000","last_active":"2024-04-25T11:42:51.581037+0000","last_peered":"2024-04-25T11:42:51.581037+0000","last_clean":"2024-04-25T11:42:51.581037+0000","last_became_active":"2024-04-25T11:42:51.580608+0000","last_became_peered":"2024-04-25T11:42:51.580608+0000","last_unstale":"2024-04-25T11:42:51.581037+0000","last_undegraded":"2024-04-25T11:42:51.581037+0000","last_fullsized":"2024-04-25T11:42:51.581037+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:13.098088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.380226+0000","last_change":"2024-04-25T11:41:44.453860+0000","last_active":"2024-04-25T11:42:51.380226+0000","last_peered":"2024-04-25T11:42:51.380226+0000","last_clean":"2024-04-25T11:42:51.380226+0000","last_became_active":"2024-04-25T11:41:44.453536+0000","last_became_peered":"2024-04-25T11:41:44.453536+0000","last_unstale":"2024-04-25T11:42:51.380226+0000","last_undegraded":"2024-04-25T11:42:51.380226+0000","last_fullsized":"2024-04-25T11:42:51.380226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:56.762115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598982+0000","last_change":"2024-04-25T11:41:44.913443+0000","last_active":"2024-04-25T11:42:51.598982+0000","last_peered":"2024-04-25T11:42:51.598982+0000","last_clean":"2024-04-25T11:42:51.598982+0000","last_became_active":"2024-04-25T11:41:44.911350+0000","last_became_peered":"2024-04-25T11:41:44.911350+0000","last_unstale":"2024-04-25T11:42:51.598982+0000","last_undegraded":"2024-04-25T11:42:51.598982+0000","last_fullsized":"2024-04-25T11:42:51.598982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:58.345587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'27","reported_seq":176,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:53.662179+0000","last_change":"2024-04-25T11:41:44.452286+0000","last_active":"2024-04-25T11:42:53.662179+0000","last_peered":"2024-04-25T11:42:53.662179+0000","last_clean":"2024-04-25T11:42:53.662179+0000","last_became_active":"2024-04-25T11:41:44.451928+0000","last_became_peered":"2024-04-25T11:41:44.451928+0000","last_unstale":"2024-04-25T11:42:53.662179+0000","last_undegraded":"2024-04-25T11:42:53.662179+0000","last_fullsized":"2024-04-25T11:42:53.662179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:21.407375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58912,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":130,"num_read_kb":123,"num_write":29,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080880+0000","last_change":"2024-04-25T11:41:44.903909+0000","last_active":"2024-04-25T11:42:51.080880+0000","last_peered":"2024-04-25T11:42:51.080880+0000","last_clean":"2024-04-25T11:42:51.080880+0000","last_became_active":"2024-04-25T11:41:44.903620+0000","last_became_peered":"2024-04-25T11:41:44.903620+0000","last_unstale":"2024-04-25T11:42:51.080880+0000","last_undegraded":"2024-04-25T11:42:51.080880+0000","last_fullsized":"2024-04-25T11:42:51.080880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:49.910052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081746+0000","last_change":"2024-04-25T11:41:44.454294+0000","last_active":"2024-04-25T11:42:51.081746+0000","last_peered":"2024-04-25T11:42:51.081746+0000","last_clean":"2024-04-25T11:42:51.081746+0000","last_became_active":"2024-04-25T11:41:44.453829+0000","last_became_peered":"2024-04-25T11:41:44.453829+0000","last_unstale":"2024-04-25T11:42:51.081746+0000","last_undegraded":"2024-04-25T11:42:51.081746+0000","last_fullsized":"2024-04-25T11:42:51.081746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:14.617963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.579841+0000","last_change":"2024-04-25T11:41:44.907453+0000","last_active":"2024-04-25T11:42:51.579841+0000","last_peered":"2024-04-25T11:42:51.579841+0000","last_clean":"2024-04-25T11:42:51.579841+0000","last_became_active":"2024-04-25T11:41:44.907126+0000","last_became_peered":"2024-04-25T11:41:44.907126+0000","last_unstale":"2024-04-25T11:42:51.579841+0000","last_undegraded":"2024-04-25T11:42:51.579841+0000","last_fullsized":"2024-04-25T11:42:51.579841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:25.998686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.379835+0000","last_change":"2024-04-25T11:41:44.452258+0000","last_active":"2024-04-25T11:42:51.379835+0000","last_peered":"2024-04-25T11:42:51.379835+0000","last_clean":"2024-04-25T11:42:51.379835+0000","last_became_active":"2024-04-25T11:41:44.451880+0000","last_became_peered":"2024-04-25T11:41:44.451880+0000","last_unstale":"2024-04-25T11:42:51.379835+0000","last_undegraded":"2024-04-25T11:42:51.379835+0000","last_fullsized":"2024-04-25T11:42:51.379835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:39.463406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080784+0000","last_change":"2024-04-25T11:41:44.913841+0000","last_active":"2024-04-25T11:42:51.080784+0000","last_peered":"2024-04-25T11:42:51.080784+0000","last_clean":"2024-04-25T11:42:51.080784+0000","last_became_active":"2024-04-25T11:41:44.913378+0000","last_became_peered":"2024-04-25T11:41:44.913378+0000","last_unstale":"2024-04-25T11:42:51.080784+0000","last_undegraded":"2024-04-25T11:42:51.080784+0000","last_fullsized":"2024-04-25T11:42:51.080784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:24.732899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:53.671833+0000","last_change":"2024-04-25T11:41:44.450531+0000","last_active":"2024-04-25T11:42:53.671833+0000","last_peered":"2024-04-25T11:42:53.671833+0000","last_clean":"2024-04-25T11:42:53.671833+0000","last_became_active":"2024-04-25T11:41:44.450205+0000","last_became_peered":"2024-04-25T11:41:44.450205+0000","last_unstale":"2024-04-25T11:42:53.671833+0000","last_undegraded":"2024-04-25T11:42:53.671833+0000","last_fullsized":"2024-04-25T11:42:53.671833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:51.797969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080312+0000","last_change":"2024-04-25T11:41:44.911883+0000","last_active":"2024-04-25T11:42:51.080312+0000","last_peered":"2024-04-25T11:42:51.080312+0000","last_clean":"2024-04-25T11:42:51.080312+0000","last_became_active":"2024-04-25T11:41:44.911362+0000","last_became_peered":"2024-04-25T11:41:44.911362+0000","last_unstale":"2024-04-25T11:42:51.080312+0000","last_undegraded":"2024-04-25T11:42:51.080312+0000","last_fullsized":"2024-04-25T11:42:51.080312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:34.273876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080169+0000","last_change":"2024-04-25T11:41:44.455844+0000","last_active":"2024-04-25T11:42:51.080169+0000","last_peered":"2024-04-25T11:42:51.080169+0000","last_clean":"2024-04-25T11:42:51.080169+0000","last_became_active":"2024-04-25T11:41:44.455167+0000","last_became_peered":"2024-04-25T11:41:44.455167+0000","last_unstale":"2024-04-25T11:42:51.080169+0000","last_undegraded":"2024-04-25T11:42:51.080169+0000","last_fullsized":"2024-04-25T11:42:51.080169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.886078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.599983+0000","last_change":"2024-04-25T11:42:51.599983+0000","last_active":"2024-04-25T11:42:51.599983+0000","last_peered":"2024-04-25T11:42:51.599983+0000","last_clean":"2024-04-25T11:42:51.599983+0000","last_became_active":"2024-04-25T11:42:51.599536+0000","last_became_peered":"2024-04-25T11:42:51.599536+0000","last_unstale":"2024-04-25T11:42:51.599983+0000","last_undegraded":"2024-04-25T11:42:51.599983+0000","last_fullsized":"2024-04-25T11:42:51.599983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:46.705406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080962+0000","last_change":"2024-04-25T11:41:44.452371+0000","last_active":"2024-04-25T11:42:51.080962+0000","last_peered":"2024-04-25T11:42:51.080962+0000","last_clean":"2024-04-25T11:42:51.080962+0000","last_became_active":"2024-04-25T11:41:44.451406+0000","last_became_peered":"2024-04-25T11:41:44.451406+0000","last_unstale":"2024-04-25T11:42:51.080962+0000","last_undegraded":"2024-04-25T11:42:51.080962+0000","last_fullsized":"2024-04-25T11:42:51.080962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:00.093835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078136+0000","last_change":"2024-04-25T11:41:44.910999+0000","last_active":"2024-04-25T11:42:51.078136+0000","last_peered":"2024-04-25T11:42:51.078136+0000","last_clean":"2024-04-25T11:42:51.078136+0000","last_became_active":"2024-04-25T11:41:44.909749+0000","last_became_peered":"2024-04-25T11:41:44.909749+0000","last_unstale":"2024-04-25T11:42:51.078136+0000","last_undegraded":"2024-04-25T11:42:51.078136+0000","last_fullsized":"2024-04-25T11:42:51.078136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:44.272145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.083142+0000","last_change":"2024-04-25T11:42:51.083142+0000","last_active":"2024-04-25T11:42:51.083142+0000","last_peered":"2024-04-25T11:42:51.083142+0000","last_clean":"2024-04-25T11:42:51.083142+0000","last_became_active":"2024-04-25T11:42:51.082717+0000","last_became_peered":"2024-04-25T11:42:51.082717+0000","last_unstale":"2024-04-25T11:42:51.083142+0000","last_undegraded":"2024-04-25T11:42:51.083142+0000","last_fullsized":"2024-04-25T11:42:51.083142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:11.639907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078611+0000","last_change":"2024-04-25T11:41:44.907350+0000","last_active":"2024-04-25T11:42:51.078611+0000","last_peered":"2024-04-25T11:42:51.078611+0000","last_clean":"2024-04-25T11:42:51.078611+0000","last_became_active":"2024-04-25T11:41:44.907000+0000","last_became_peered":"2024-04-25T11:41:44.907000+0000","last_unstale":"2024-04-25T11:42:51.078611+0000","last_undegraded":"2024-04-25T11:42:51.078611+0000","last_fullsized":"2024-04-25T11:42:51.078611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:08.823550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":89,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:53.671524+0000","last_change":"2024-04-25T11:41:44.455318+0000","last_active":"2024-04-25T11:42:53.671524+0000","last_peered":"2024-04-25T11:42:53.671524+0000","last_clean":"2024-04-25T11:42:53.671524+0000","last_became_active":"2024-04-25T11:41:44.454972+0000","last_became_peered":"2024-04-25T11:41:44.454972+0000","last_unstale":"2024-04-25T11:42:53.671524+0000","last_undegraded":"2024-04-25T11:42:53.671524+0000","last_fullsized":"2024-04-25T11:42:53.671524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.255860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078641+0000","last_change":"2024-04-25T11:41:44.912507+0000","last_active":"2024-04-25T11:42:51.078641+0000","last_peered":"2024-04-25T11:42:51.078641+0000","last_clean":"2024-04-25T11:42:51.078641+0000","last_became_active":"2024-04-25T11:41:44.912123+0000","last_became_peered":"2024-04-25T11:41:44.912123+0000","last_unstale":"2024-04-25T11:42:51.078641+0000","last_undegraded":"2024-04-25T11:42:51.078641+0000","last_fullsized":"2024-04-25T11:42:51.078641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:40.659961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.379732+0000","last_change":"2024-04-25T11:41:44.913505+0000","last_active":"2024-04-25T11:42:51.379732+0000","last_peered":"2024-04-25T11:42:51.379732+0000","last_clean":"2024-04-25T11:42:51.379732+0000","last_became_active":"2024-04-25T11:41:44.913103+0000","last_became_peered":"2024-04-25T11:41:44.913103+0000","last_unstale":"2024-04-25T11:42:51.379732+0000","last_undegraded":"2024-04-25T11:42:51.379732+0000","last_fullsized":"2024-04-25T11:42:51.379732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:49.414802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":89,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:53.671286+0000","last_change":"2024-04-25T11:41:44.450056+0000","last_active":"2024-04-25T11:42:53.671286+0000","last_peered":"2024-04-25T11:42:53.671286+0000","last_clean":"2024-04-25T11:42:53.671286+0000","last_became_active":"2024-04-25T11:41:44.449754+0000","last_became_peered":"2024-04-25T11:41:44.449754+0000","last_unstale":"2024-04-25T11:42:53.671286+0000","last_undegraded":"2024-04-25T11:42:53.671286+0000","last_fullsized":"2024-04-25T11:42:53.671286+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:30.059852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598221+0000","last_change":"2024-04-25T11:41:44.915061+0000","last_active":"2024-04-25T11:42:51.598221+0000","last_peered":"2024-04-25T11:42:51.598221+0000","last_clean":"2024-04-25T11:42:51.598221+0000","last_became_active":"2024-04-25T11:41:44.914799+0000","last_became_peered":"2024-04-25T11:41:44.914799+0000","last_unstale":"2024-04-25T11:42:51.598221+0000","last_undegraded":"2024-04-25T11:42:51.598221+0000","last_fullsized":"2024-04-25T11:42:51.598221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:22.820539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.599445+0000","last_change":"2024-04-25T11:41:44.449248+0000","last_active":"2024-04-25T11:42:51.599445+0000","last_peered":"2024-04-25T11:42:51.599445+0000","last_clean":"2024-04-25T11:42:51.599445+0000","last_became_active":"2024-04-25T11:41:44.448898+0000","last_became_peered":"2024-04-25T11:41:44.448898+0000","last_unstale":"2024-04-25T11:42:51.599445+0000","last_undegraded":"2024-04-25T11:42:51.599445+0000","last_fullsized":"2024-04-25T11:42:51.599445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:19.574270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080486+0000","last_change":"2024-04-25T11:40:36.892561+0000","last_active":"2024-04-25T11:42:51.080486+0000","last_peered":"2024-04-25T11:42:51.080486+0000","last_clean":"2024-04-25T11:42:51.080486+0000","last_became_active":"2024-04-25T11:40:36.892126+0000","last_became_peered":"2024-04-25T11:40:36.892126+0000","last_unstale":"2024-04-25T11:42:51.080486+0000","last_undegraded":"2024-04-25T11:42:51.080486+0000","last_fullsized":"2024-04-25T11:42:51.080486+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:37:16.666579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:37:16.666579+0000","last_clean_scrub_stamp":"2024-04-25T11:37:16.666579+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:44.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.579703+0000","last_change":"2024-04-25T11:41:44.908446+0000","last_active":"2024-04-25T11:42:51.579703+0000","last_peered":"2024-04-25T11:42:51.579703+0000","last_clean":"2024-04-25T11:42:51.579703+0000","last_became_active":"2024-04-25T11:41:44.908148+0000","last_became_peered":"2024-04-25T11:41:44.908148+0000","last_unstale":"2024-04-25T11:42:51.579703+0000","last_undegraded":"2024-04-25T11:42:51.579703+0000","last_fullsized":"2024-04-25T11:42:51.579703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:09.598976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080643+0000","last_change":"2024-04-25T11:41:44.454815+0000","last_active":"2024-04-25T11:42:51.080643+0000","last_peered":"2024-04-25T11:42:51.080643+0000","last_clean":"2024-04-25T11:42:51.080643+0000","last_became_active":"2024-04-25T11:41:44.454521+0000","last_became_peered":"2024-04-25T11:41:44.454521+0000","last_unstale":"2024-04-25T11:42:51.080643+0000","last_undegraded":"2024-04-25T11:42:51.080643+0000","last_fullsized":"2024-04-25T11:42:51.080643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:45.932329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081011+0000","last_change":"2024-04-25T11:41:44.456641+0000","last_active":"2024-04-25T11:42:51.081011+0000","last_peered":"2024-04-25T11:42:51.081011+0000","last_clean":"2024-04-25T11:42:51.081011+0000","last_became_active":"2024-04-25T11:41:44.456133+0000","last_became_peered":"2024-04-25T11:41:44.456133+0000","last_unstale":"2024-04-25T11:42:51.081011+0000","last_undegraded":"2024-04-25T11:42:51.081011+0000","last_fullsized":"2024-04-25T11:42:51.081011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:47.746160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.580086+0000","last_change":"2024-04-25T11:41:44.897512+0000","last_active":"2024-04-25T11:42:51.580086+0000","last_peered":"2024-04-25T11:42:51.580086+0000","last_clean":"2024-04-25T11:42:51.580086+0000","last_became_active":"2024-04-25T11:41:44.897184+0000","last_became_peered":"2024-04-25T11:41:44.897184+0000","last_unstale":"2024-04-25T11:42:51.580086+0000","last_undegraded":"2024-04-25T11:42:51.580086+0000","last_fullsized":"2024-04-25T11:42:51.580086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:53.703825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078338+0000","last_change":"2024-04-25T11:41:44.458318+0000","last_active":"2024-04-25T11:42:51.078338+0000","last_peered":"2024-04-25T11:42:51.078338+0000","last_clean":"2024-04-25T11:42:51.078338+0000","last_became_active":"2024-04-25T11:41:44.458001+0000","last_became_peered":"2024-04-25T11:41:44.458001+0000","last_unstale":"2024-04-25T11:42:51.078338+0000","last_undegraded":"2024-04-25T11:42:51.078338+0000","last_fullsized":"2024-04-25T11:42:51.078338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:17.100621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081876+0000","last_change":"2024-04-25T11:42:51.081876+0000","last_active":"2024-04-25T11:42:51.081876+0000","last_peered":"2024-04-25T11:42:51.081876+0000","last_clean":"2024-04-25T11:42:51.081876+0000","last_became_active":"2024-04-25T11:42:51.081566+0000","last_became_peered":"2024-04-25T11:42:51.081566+0000","last_unstale":"2024-04-25T11:42:51.081876+0000","last_undegraded":"2024-04-25T11:42:51.081876+0000","last_fullsized":"2024-04-25T11:42:51.081876+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:05.530080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.599402+0000","last_change":"2024-04-25T11:41:44.460492+0000","last_active":"2024-04-25T11:42:51.599402+0000","last_peered":"2024-04-25T11:42:51.599402+0000","last_clean":"2024-04-25T11:42:51.599402+0000","last_became_active":"2024-04-25T11:41:44.460199+0000","last_became_peered":"2024-04-25T11:41:44.460199+0000","last_unstale":"2024-04-25T11:42:51.599402+0000","last_undegraded":"2024-04-25T11:42:51.599402+0000","last_fullsized":"2024-04-25T11:42:51.599402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:18.756779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080394+0000","last_change":"2024-04-25T11:41:44.909438+0000","last_active":"2024-04-25T11:42:51.080394+0000","last_peered":"2024-04-25T11:42:51.080394+0000","last_clean":"2024-04-25T11:42:51.080394+0000","last_became_active":"2024-04-25T11:41:44.909146+0000","last_became_peered":"2024-04-25T11:41:44.909146+0000","last_unstale":"2024-04-25T11:42:51.080394+0000","last_undegraded":"2024-04-25T11:42:51.080394+0000","last_fullsized":"2024-04-25T11:42:51.080394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:15.949652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598415+0000","last_change":"2024-04-25T11:41:44.453169+0000","last_active":"2024-04-25T11:42:51.598415+0000","last_peered":"2024-04-25T11:42:51.598415+0000","last_clean":"2024-04-25T11:42:51.598415+0000","last_became_active":"2024-04-25T11:41:44.452762+0000","last_became_peered":"2024-04-25T11:41:44.452762+0000","last_unstale":"2024-04-25T11:42:51.598415+0000","last_undegraded":"2024-04-25T11:42:51.598415+0000","last_fullsized":"2024-04-25T11:42:51.598415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:13.084197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.079807+0000","last_change":"2024-04-25T11:41:44.910810+0000","last_active":"2024-04-25T11:42:51.079807+0000","last_peered":"2024-04-25T11:42:51.079807+0000","last_clean":"2024-04-25T11:42:51.079807+0000","last_became_active":"2024-04-25T11:41:44.910488+0000","last_became_peered":"2024-04-25T11:41:44.910488+0000","last_unstale":"2024-04-25T11:42:51.079807+0000","last_undegraded":"2024-04-25T11:42:51.079807+0000","last_fullsized":"2024-04-25T11:42:51.079807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:36.971289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598001+0000","last_change":"2024-04-25T11:41:44.455402+0000","last_active":"2024-04-25T11:42:51.598001+0000","last_peered":"2024-04-25T11:42:51.598001+0000","last_clean":"2024-04-25T11:42:51.598001+0000","last_became_active":"2024-04-25T11:41:44.455057+0000","last_became_peered":"2024-04-25T11:41:44.455057+0000","last_unstale":"2024-04-25T11:42:51.598001+0000","last_undegraded":"2024-04-25T11:42:51.598001+0000","last_fullsized":"2024-04-25T11:42:51.598001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:32.156664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.077934+0000","last_change":"2024-04-25T11:41:44.903907+0000","last_active":"2024-04-25T11:42:51.077934+0000","last_peered":"2024-04-25T11:42:51.077934+0000","last_clean":"2024-04-25T11:42:51.077934+0000","last_became_active":"2024-04-25T11:41:44.903510+0000","last_became_peered":"2024-04-25T11:41:44.903510+0000","last_unstale":"2024-04-25T11:42:51.077934+0000","last_undegraded":"2024-04-25T11:42:51.077934+0000","last_fullsized":"2024-04-25T11:42:51.077934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:33.183412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081108+0000","last_change":"2024-04-25T11:41:44.454049+0000","last_active":"2024-04-25T11:42:51.081108+0000","last_peered":"2024-04-25T11:42:51.081108+0000","last_clean":"2024-04-25T11:42:51.081108+0000","last_became_active":"2024-04-25T11:41:44.453652+0000","last_became_peered":"2024-04-25T11:41:44.453652+0000","last_unstale":"2024-04-25T11:42:51.081108+0000","last_undegraded":"2024-04-25T11:42:51.081108+0000","last_fullsized":"2024-04-25T11:42:51.081108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:17.239715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598209+0000","last_change":"2024-04-25T11:41:44.915005+0000","last_active":"2024-04-25T11:42:51.598209+0000","last_peered":"2024-04-25T11:42:51.598209+0000","last_clean":"2024-04-25T11:42:51.598209+0000","last_became_active":"2024-04-25T11:41:44.914663+0000","last_became_peered":"2024-04-25T11:41:44.914663+0000","last_unstale":"2024-04-25T11:42:51.598209+0000","last_undegraded":"2024-04-25T11:42:51.598209+0000","last_fullsized":"2024-04-25T11:42:51.598209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:49.390846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080022+0000","last_change":"2024-04-25T11:41:44.454194+0000","last_active":"2024-04-25T11:42:51.080022+0000","last_peered":"2024-04-25T11:42:51.080022+0000","last_clean":"2024-04-25T11:42:51.080022+0000","last_became_active":"2024-04-25T11:41:44.453280+0000","last_became_peered":"2024-04-25T11:41:44.453280+0000","last_unstale":"2024-04-25T11:42:51.080022+0000","last_undegraded":"2024-04-25T11:42:51.080022+0000","last_fullsized":"2024-04-25T11:42:51.080022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:45.368132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080654+0000","last_change":"2024-04-25T11:41:44.907951+0000","last_active":"2024-04-25T11:42:51.080654+0000","last_peered":"2024-04-25T11:42:51.080654+0000","last_clean":"2024-04-25T11:42:51.080654+0000","last_became_active":"2024-04-25T11:41:44.907554+0000","last_became_peered":"2024-04-25T11:41:44.907554+0000","last_unstale":"2024-04-25T11:42:51.080654+0000","last_undegraded":"2024-04-25T11:42:51.080654+0000","last_fullsized":"2024-04-25T11:42:51.080654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:33.312265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:55.668728+0000","last_change":"2024-04-25T11:41:44.458803+0000","last_active":"2024-04-25T11:42:55.668728+0000","last_peered":"2024-04-25T11:42:55.668728+0000","last_clean":"2024-04-25T11:42:55.668728+0000","last_became_active":"2024-04-25T11:41:44.458307+0000","last_became_peered":"2024-04-25T11:41:44.458307+0000","last_unstale":"2024-04-25T11:42:55.668728+0000","last_undegraded":"2024-04-25T11:42:55.668728+0000","last_fullsized":"2024-04-25T11:42:55.668728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:26.186429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081013+0000","last_change":"2024-04-25T11:41:44.909600+0000","last_active":"2024-04-25T11:42:51.081013+0000","last_peered":"2024-04-25T11:42:51.081013+0000","last_clean":"2024-04-25T11:42:51.081013+0000","last_became_active":"2024-04-25T11:41:44.909083+0000","last_became_peered":"2024-04-25T11:41:44.909083+0000","last_unstale":"2024-04-25T11:42:51.081013+0000","last_undegraded":"2024-04-25T11:42:51.081013+0000","last_fullsized":"2024-04-25T11:42:51.081013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:53.854631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.380077+0000","last_change":"2024-04-25T11:41:44.451466+0000","last_active":"2024-04-25T11:42:51.380077+0000","last_peered":"2024-04-25T11:42:51.380077+0000","last_clean":"2024-04-25T11:42:51.380077+0000","last_became_active":"2024-04-25T11:41:44.450866+0000","last_became_peered":"2024-04-25T11:41:44.450866+0000","last_unstale":"2024-04-25T11:42:51.380077+0000","last_undegraded":"2024-04-25T11:42:51.380077+0000","last_fullsized":"2024-04-25T11:42:51.380077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:03.715468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080753+0000","last_change":"2024-04-25T11:41:44.909789+0000","last_active":"2024-04-25T11:42:51.080753+0000","last_peered":"2024-04-25T11:42:51.080753+0000","last_clean":"2024-04-25T11:42:51.080753+0000","last_became_active":"2024-04-25T11:41:44.909490+0000","last_became_peered":"2024-04-25T11:41:44.909490+0000","last_unstale":"2024-04-25T11:42:51.080753+0000","last_undegraded":"2024-04-25T11:42:51.080753+0000","last_fullsized":"2024-04-25T11:42:51.080753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:50.543847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.580097+0000","last_change":"2024-04-25T11:41:44.449371+0000","last_active":"2024-04-25T11:42:51.580097+0000","last_peered":"2024-04-25T11:42:51.580097+0000","last_clean":"2024-04-25T11:42:51.580097+0000","last_became_active":"2024-04-25T11:41:44.448927+0000","last_became_peered":"2024-04-25T11:41:44.448927+0000","last_unstale":"2024-04-25T11:42:51.580097+0000","last_undegraded":"2024-04-25T11:42:51.580097+0000","last_fullsized":"2024-04-25T11:42:51.580097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:49.581010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081092+0000","last_change":"2024-04-25T11:41:44.906540+0000","last_active":"2024-04-25T11:42:51.081092+0000","last_peered":"2024-04-25T11:42:51.081092+0000","last_clean":"2024-04-25T11:42:51.081092+0000","last_became_active":"2024-04-25T11:41:44.905918+0000","last_became_peered":"2024-04-25T11:41:44.905918+0000","last_unstale":"2024-04-25T11:42:51.081092+0000","last_undegraded":"2024-04-25T11:42:51.081092+0000","last_fullsized":"2024-04-25T11:42:51.081092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:38.350461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078014+0000","last_change":"2024-04-25T11:41:44.904656+0000","last_active":"2024-04-25T11:42:51.078014+0000","last_peered":"2024-04-25T11:42:51.078014+0000","last_clean":"2024-04-25T11:42:51.078014+0000","last_became_active":"2024-04-25T11:41:44.904322+0000","last_became_peered":"2024-04-25T11:41:44.904322+0000","last_unstale":"2024-04-25T11:42:51.078014+0000","last_undegraded":"2024-04-25T11:42:51.078014+0000","last_fullsized":"2024-04-25T11:42:51.078014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:07.762816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081934+0000","last_change":"2024-04-25T11:41:44.911662+0000","last_active":"2024-04-25T11:42:51.081934+0000","last_peered":"2024-04-25T11:42:51.081934+0000","last_clean":"2024-04-25T11:42:51.081934+0000","last_became_active":"2024-04-25T11:41:44.911295+0000","last_became_peered":"2024-04-25T11:41:44.911295+0000","last_unstale":"2024-04-25T11:42:51.081934+0000","last_undegraded":"2024-04-25T11:42:51.081934+0000","last_fullsized":"2024-04-25T11:42:51.081934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:12.988118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080707+0000","last_change":"2024-04-25T11:41:44.906303+0000","last_active":"2024-04-25T11:42:51.080707+0000","last_peered":"2024-04-25T11:42:51.080707+0000","last_clean":"2024-04-25T11:42:51.080707+0000","last_became_active":"2024-04-25T11:41:44.905951+0000","last_became_peered":"2024-04-25T11:41:44.905951+0000","last_unstale":"2024-04-25T11:42:51.080707+0000","last_undegraded":"2024-04-25T11:42:51.080707+0000","last_fullsized":"2024-04-25T11:42:51.080707+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:49.887856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.580929+0000","last_change":"2024-04-25T11:42:51.580929+0000","last_active":"2024-04-25T11:42:51.580929+0000","last_peered":"2024-04-25T11:42:51.580929+0000","last_clean":"2024-04-25T11:42:51.580929+0000","last_became_active":"2024-04-25T11:42:51.580492+0000","last_became_peered":"2024-04-25T11:42:51.580492+0000","last_unstale":"2024-04-25T11:42:51.580929+0000","last_undegraded":"2024-04-25T11:42:51.580929+0000","last_fullsized":"2024-04-25T11:42:51.580929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:16.272379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.579960+0000","last_change":"2024-04-25T11:41:44.909391+0000","last_active":"2024-04-25T11:42:51.579960+0000","last_peered":"2024-04-25T11:42:51.579960+0000","last_clean":"2024-04-25T11:42:51.579960+0000","last_became_active":"2024-04-25T11:41:44.909088+0000","last_became_peered":"2024-04-25T11:41:44.909088+0000","last_unstale":"2024-04-25T11:42:51.579960+0000","last_undegraded":"2024-04-25T11:42:51.579960+0000","last_fullsized":"2024-04-25T11:42:51.579960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:38.550716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078275+0000","last_change":"2024-04-25T11:41:44.904351+0000","last_active":"2024-04-25T11:42:51.078275+0000","last_peered":"2024-04-25T11:42:51.078275+0000","last_clean":"2024-04-25T11:42:51.078275+0000","last_became_active":"2024-04-25T11:41:44.904015+0000","last_became_peered":"2024-04-25T11:41:44.904015+0000","last_unstale":"2024-04-25T11:42:51.078275+0000","last_undegraded":"2024-04-25T11:42:51.078275+0000","last_fullsized":"2024-04-25T11:42:51.078275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:18.382370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081641+0000","last_change":"2024-04-25T11:41:44.901469+0000","last_active":"2024-04-25T11:42:51.081641+0000","last_peered":"2024-04-25T11:42:51.081641+0000","last_clean":"2024-04-25T11:42:51.081641+0000","last_became_active":"2024-04-25T11:41:44.900943+0000","last_became_peered":"2024-04-25T11:41:44.900943+0000","last_unstale":"2024-04-25T11:42:51.081641+0000","last_undegraded":"2024-04-25T11:42:51.081641+0000","last_fullsized":"2024-04-25T11:42:51.081641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:33.712132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598326+0000","last_change":"2024-04-25T11:41:44.912267+0000","last_active":"2024-04-25T11:42:51.598326+0000","last_peered":"2024-04-25T11:42:51.598326+0000","last_clean":"2024-04-25T11:42:51.598326+0000","last_became_active":"2024-04-25T11:41:44.911968+0000","last_became_peered":"2024-04-25T11:41:44.911968+0000","last_unstale":"2024-04-25T11:42:51.598326+0000","last_undegraded":"2024-04-25T11:42:51.598326+0000","last_fullsized":"2024-04-25T11:42:51.598326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:27.956046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.379651+0000","last_change":"2024-04-25T11:41:44.907540+0000","last_active":"2024-04-25T11:42:51.379651+0000","last_peered":"2024-04-25T11:42:51.379651+0000","last_clean":"2024-04-25T11:42:51.379651+0000","last_became_active":"2024-04-25T11:41:44.907229+0000","last_became_peered":"2024-04-25T11:41:44.907229+0000","last_unstale":"2024-04-25T11:42:51.379651+0000","last_undegraded":"2024-04-25T11:42:51.379651+0000","last_fullsized":"2024-04-25T11:42:51.379651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:49.123196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.597941+0000","last_change":"2024-04-25T11:41:44.912354+0000","last_active":"2024-04-25T11:42:51.597941+0000","last_peered":"2024-04-25T11:42:51.597941+0000","last_clean":"2024-04-25T11:42:51.597941+0000","last_became_active":"2024-04-25T11:41:44.912028+0000","last_became_peered":"2024-04-25T11:41:44.912028+0000","last_unstale":"2024-04-25T11:42:51.597941+0000","last_undegraded":"2024-04-25T11:42:51.597941+0000","last_fullsized":"2024-04-25T11:42:51.597941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:50.824647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598906+0000","last_change":"2024-04-25T11:41:44.907346+0000","last_active":"2024-04-25T11:42:51.598906+0000","last_peered":"2024-04-25T11:42:51.598906+0000","last_clean":"2024-04-25T11:42:51.598906+0000","last_became_active":"2024-04-25T11:41:44.907027+0000","last_became_peered":"2024-04-25T11:41:44.907027+0000","last_unstale":"2024-04-25T11:42:51.598906+0000","last_undegraded":"2024-04-25T11:42:51.598906+0000","last_fullsized":"2024-04-25T11:42:51.598906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:01.913218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.580008+0000","last_change":"2024-04-25T11:41:44.902606+0000","last_active":"2024-04-25T11:42:51.580008+0000","last_peered":"2024-04-25T11:42:51.580008+0000","last_clean":"2024-04-25T11:42:51.580008+0000","last_became_active":"2024-04-25T11:41:44.902306+0000","last_became_peered":"2024-04-25T11:41:44.902306+0000","last_unstale":"2024-04-25T11:42:51.580008+0000","last_undegraded":"2024-04-25T11:42:51.580008+0000","last_fullsized":"2024-04-25T11:42:51.580008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:24.214013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.083688+0000","last_change":"2024-04-25T11:42:51.083688+0000","last_active":"2024-04-25T11:42:51.083688+0000","last_peered":"2024-04-25T11:42:51.083688+0000","last_clean":"2024-04-25T11:42:51.083688+0000","last_became_active":"2024-04-25T11:42:51.082806+0000","last_became_peered":"2024-04-25T11:42:51.082806+0000","last_unstale":"2024-04-25T11:42:51.083688+0000","last_undegraded":"2024-04-25T11:42:51.083688+0000","last_fullsized":"2024-04-25T11:42:51.083688+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:03.684385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66116,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":669,"num_read_kb":405,"num_write":234,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":198348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351002,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":460429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612640,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1658880,"data_stored":1110761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776166,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":460359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037801,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":460737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299439,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708268,"statfs":{"total":95995035648,"available":95957266432,"internally_reserved":0,"allocated":1576960,"data_stored":1053646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462964,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":966656,"data_stored":460076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":519901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":461415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":520512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":459901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888126,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":460328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1589248,"data_stored":1052933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:42:59.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T11:42:59.929 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:43:00.363 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:00 smithi083 ceph-mon[29330]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T11:43:00.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:00 smithi094 ceph-mon[33032]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T11:43:00.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:00 smithi111 ceph-mon[32784]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T11:43:01.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.415+0000 7fd916e1d700 1 -- 172.21.15.83:0/1488930857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd910101940 msgr2=0x7fd910101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:01.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.415+0000 7fd916e1d700 1 --2- 172.21.15.83:0/1488930857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd910101940 0x7fd910101dc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd904009990 tx=0x7fd904009ca0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.415+0000 7fd916e1d700 1 -- 172.21.15.83:0/1488930857 shutdown_connections 2024-04-25T11:43:01.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.415+0000 7fd916e1d700 1 --2- 172.21.15.83:0/1488930857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd910102300 0x7fd91010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.415+0000 7fd916e1d700 1 --2- 172.21.15.83:0/1488930857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd910107990 0x7fd910107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.415+0000 7fd916e1d700 1 --2- 172.21.15.83:0/1488930857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd910101940 0x7fd910101dc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.415+0000 7fd916e1d700 1 -- 172.21.15.83:0/1488930857 >> 172.21.15.83:0/1488930857 conn(0x7fd9100788f0 msgr2=0x7fd910078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:01.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.415+0000 7fd916e1d700 1 -- 172.21.15.83:0/1488930857 shutdown_connections 2024-04-25T11:43:01.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.415+0000 7fd916e1d700 1 -- 172.21.15.83:0/1488930857 wait complete. 2024-04-25T11:43:01.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.416+0000 7fd916e1d700 1 Processor -- start 2024-04-25T11:43:01.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.416+0000 7fd916e1d700 1 -- start start 2024-04-25T11:43:01.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.416+0000 7fd916e1d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd910101940 0x7fd910199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:01.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.416+0000 7fd916e1d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd910102300 0x7fd91019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:01.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.416+0000 7fd916e1d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd910107990 0x7fd9101a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:01.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.416+0000 7fd916e1d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd910111b70 con 0x7fd910102300 2024-04-25T11:43:01.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.416+0000 7fd916e1d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101119f0 con 0x7fd910107990 2024-04-25T11:43:01.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.416+0000 7fd916e1d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd910111cf0 con 0x7fd910101940 2024-04-25T11:43:01.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.416+0000 7fd90ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd910102300 0x7fd91019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:01.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.416+0000 7fd90ffff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd910102300 0x7fd91019a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:41606/0 (socket says 172.21.15.83:41606) 2024-04-25T11:43:01.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.416+0000 7fd90ffff700 1 -- 172.21.15.83:0/898509964 learned_addr learned my addr 172.21.15.83:0/898509964 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:01.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.416+0000 7fd914bb9700 1 --2- 172.21.15.83:0/898509964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd910101940 0x7fd910199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:01.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.416+0000 7fd9153ba700 1 --2- 172.21.15.83:0/898509964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd910107990 0x7fd9101a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:01.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.417+0000 7fd90ffff700 1 -- 172.21.15.83:0/898509964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd910101940 msgr2=0x7fd910199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:01.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.417+0000 7fd90ffff700 1 --2- 172.21.15.83:0/898509964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd910101940 0x7fd910199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.417+0000 7fd90ffff700 1 -- 172.21.15.83:0/898509964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd910107990 msgr2=0x7fd9101a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:01.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.417+0000 7fd90ffff700 1 --2- 172.21.15.83:0/898509964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd910107990 0x7fd9101a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.417+0000 7fd90ffff700 1 -- 172.21.15.83:0/898509964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9101a55b0 con 0x7fd910102300 2024-04-25T11:43:01.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.417+0000 7fd9153ba700 1 --2- 172.21.15.83:0/898509964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd910107990 0x7fd9101a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:01.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.417+0000 7fd90ffff700 1 --2- 172.21.15.83:0/898509964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd910102300 0x7fd91019a530 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd904000c00 tx=0x7fd9040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:01.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.417+0000 7fd90dffb700 1 -- 172.21.15.83:0/898509964 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904027070 con 0x7fd910102300 2024-04-25T11:43:01.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.417+0000 7fd916e1d700 1 -- 172.21.15.83:0/898509964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9101a5780 con 0x7fd910102300 2024-04-25T11:43:01.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.417+0000 7fd90dffb700 1 -- 172.21.15.83:0/898509964 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd904003f20 con 0x7fd910102300 2024-04-25T11:43:01.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.417+0000 7fd90dffb700 1 -- 172.21.15.83:0/898509964 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904022cf0 con 0x7fd910102300 2024-04-25T11:43:01.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.417+0000 7fd916e1d700 1 -- 172.21.15.83:0/898509964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9101a5bd0 con 0x7fd910102300 2024-04-25T11:43:01.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.418+0000 7fd916e1d700 1 -- 172.21.15.83:0/898509964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd910109eb0 con 0x7fd910102300 2024-04-25T11:43:01.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.419+0000 7fd90dffb700 1 -- 172.21.15.83:0/898509964 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd90402a070 con 0x7fd910102300 2024-04-25T11:43:01.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.420+0000 7fd90dffb700 1 --2- 172.21.15.83:0/898509964 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd8f80a3cd0 0x7fd8f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:01.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.420+0000 7fd914bb9700 1 --2- 172.21.15.83:0/898509964 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd8f80a3cd0 0x7fd8f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:01.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.420+0000 7fd90dffb700 1 -- 172.21.15.83:0/898509964 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd9040d16d0 con 0x7fd910102300 2024-04-25T11:43:01.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.420+0000 7fd914bb9700 1 --2- 172.21.15.83:0/898509964 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd8f80a3cd0 0x7fd8f80a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd908003f90 tx=0x7fd908009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:01.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.424+0000 7fd90dffb700 1 -- 172.21.15.83:0/898509964 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd904013070 con 0x7fd910102300 2024-04-25T11:43:01.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.618+0000 7fd916e1d700 1 -- 172.21.15.83:0/898509964 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd910062e80 con 0x7fd8f80a3cd0 2024-04-25T11:43:01.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.625+0000 7fd90dffb700 1 -- 172.21.15.83:0/898509964 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258743 (secure 0 0 0) 0x7fd910062e80 con 0x7fd8f80a3cd0 2024-04-25T11:43:01.616 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:01.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.628+0000 7fd916e1d700 1 -- 172.21.15.83:0/898509964 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd8f80a3cd0 msgr2=0x7fd8f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:01.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.628+0000 7fd916e1d700 1 --2- 172.21.15.83:0/898509964 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd8f80a3cd0 0x7fd8f80a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd908003f90 tx=0x7fd908009180 comp rx=0 tx=0).stop 2024-04-25T11:43:01.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.628+0000 7fd916e1d700 1 -- 172.21.15.83:0/898509964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd910102300 msgr2=0x7fd91019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:01.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.628+0000 7fd916e1d700 1 --2- 172.21.15.83:0/898509964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd910102300 0x7fd91019a530 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd904000c00 tx=0x7fd9040047b0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.628+0000 7fd916e1d700 1 -- 172.21.15.83:0/898509964 shutdown_connections 2024-04-25T11:43:01.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.628+0000 7fd916e1d700 1 --2- 172.21.15.83:0/898509964 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd8f80a3cd0 0x7fd8f80a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.628+0000 7fd916e1d700 1 --2- 172.21.15.83:0/898509964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd910107990 0x7fd9101a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.628+0000 7fd916e1d700 1 --2- 172.21.15.83:0/898509964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd910101940 0x7fd910199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.628+0000 7fd916e1d700 1 --2- 172.21.15.83:0/898509964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd910102300 0x7fd91019a530 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:01.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.628+0000 7fd916e1d700 1 -- 172.21.15.83:0/898509964 >> 172.21.15.83:0/898509964 conn(0x7fd9100788f0 msgr2=0x7fd9100ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:01.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.628+0000 7fd916e1d700 1 -- 172.21.15.83:0/898509964 shutdown_connections 2024-04-25T11:43:01.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:01.628+0000 7fd916e1d700 1 -- 172.21.15.83:0/898509964 wait complete. 2024-04-25T11:43:01.618 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-25T11:43:02.328 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-04-25T11:42:59.926715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656718,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":840,"num_read_kb":544,"num_write":417,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":342,"ondisk_log_size":342,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":294,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375116,"kb_used_data":13404,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566708,"statfs":{"total":1151940427776,"available":1151556308992,"internally_reserved":0,"allocated":13725696,"data_stored":7480998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":48,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.851364"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078580+0000","last_change":"2024-04-25T11:41:44.907326+0000","last_active":"2024-04-25T11:42:51.078580+0000","last_peered":"2024-04-25T11:42:51.078580+0000","last_clean":"2024-04-25T11:42:51.078580+0000","last_became_active":"2024-04-25T11:41:44.906964+0000","last_became_peered":"2024-04-25T11:41:44.906964+0000","last_unstale":"2024-04-25T11:42:51.078580+0000","last_undegraded":"2024-04-25T11:42:51.078580+0000","last_fullsized":"2024-04-25T11:42:51.078580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:46.656252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080527+0000","last_change":"2024-04-25T11:41:44.907601+0000","last_active":"2024-04-25T11:42:51.080527+0000","last_peered":"2024-04-25T11:42:51.080527+0000","last_clean":"2024-04-25T11:42:51.080527+0000","last_became_active":"2024-04-25T11:41:44.907142+0000","last_became_peered":"2024-04-25T11:41:44.907142+0000","last_unstale":"2024-04-25T11:42:51.080527+0000","last_undegraded":"2024-04-25T11:42:51.080527+0000","last_fullsized":"2024-04-25T11:42:51.080527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:41.901259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080315+0000","last_change":"2024-04-25T11:41:44.910479+0000","last_active":"2024-04-25T11:42:51.080315+0000","last_peered":"2024-04-25T11:42:51.080315+0000","last_clean":"2024-04-25T11:42:51.080315+0000","last_became_active":"2024-04-25T11:41:44.909711+0000","last_became_peered":"2024-04-25T11:41:44.909711+0000","last_unstale":"2024-04-25T11:42:51.080315+0000","last_undegraded":"2024-04-25T11:42:51.080315+0000","last_fullsized":"2024-04-25T11:42:51.080315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:07.586793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.599617+0000","last_change":"2024-04-25T11:41:44.909936+0000","last_active":"2024-04-25T11:42:51.599617+0000","last_peered":"2024-04-25T11:42:51.599617+0000","last_clean":"2024-04-25T11:42:51.599617+0000","last_became_active":"2024-04-25T11:41:44.909645+0000","last_became_peered":"2024-04-25T11:41:44.909645+0000","last_unstale":"2024-04-25T11:42:51.599617+0000","last_undegraded":"2024-04-25T11:42:51.599617+0000","last_fullsized":"2024-04-25T11:42:51.599617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:12.518017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080175+0000","last_change":"2024-04-25T11:41:44.913907+0000","last_active":"2024-04-25T11:42:51.080175+0000","last_peered":"2024-04-25T11:42:51.080175+0000","last_clean":"2024-04-25T11:42:51.080175+0000","last_became_active":"2024-04-25T11:41:44.912593+0000","last_became_peered":"2024-04-25T11:41:44.912593+0000","last_unstale":"2024-04-25T11:42:51.080175+0000","last_undegraded":"2024-04-25T11:42:51.080175+0000","last_fullsized":"2024-04-25T11:42:51.080175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:40.164688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078266+0000","last_change":"2024-04-25T11:41:44.903237+0000","last_active":"2024-04-25T11:42:51.078266+0000","last_peered":"2024-04-25T11:42:51.078266+0000","last_clean":"2024-04-25T11:42:51.078266+0000","last_became_active":"2024-04-25T11:41:44.902881+0000","last_became_peered":"2024-04-25T11:41:44.902881+0000","last_unstale":"2024-04-25T11:42:51.078266+0000","last_undegraded":"2024-04-25T11:42:51.078266+0000","last_fullsized":"2024-04-25T11:42:51.078266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:36.201322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080392+0000","last_change":"2024-04-25T11:41:44.907497+0000","last_active":"2024-04-25T11:42:51.080392+0000","last_peered":"2024-04-25T11:42:51.080392+0000","last_clean":"2024-04-25T11:42:51.080392+0000","last_became_active":"2024-04-25T11:41:44.907117+0000","last_became_peered":"2024-04-25T11:41:44.907117+0000","last_unstale":"2024-04-25T11:42:51.080392+0000","last_undegraded":"2024-04-25T11:42:51.080392+0000","last_fullsized":"2024-04-25T11:42:51.080392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:25.987238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.597914+0000","last_change":"2024-04-25T11:41:44.915499+0000","last_active":"2024-04-25T11:42:51.597914+0000","last_peered":"2024-04-25T11:42:51.597914+0000","last_clean":"2024-04-25T11:42:51.597914+0000","last_became_active":"2024-04-25T11:41:44.915221+0000","last_became_peered":"2024-04-25T11:41:44.915221+0000","last_unstale":"2024-04-25T11:42:51.597914+0000","last_undegraded":"2024-04-25T11:42:51.597914+0000","last_fullsized":"2024-04-25T11:42:51.597914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:36.717428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598840+0000","last_change":"2024-04-25T11:41:44.910599+0000","last_active":"2024-04-25T11:42:51.598840+0000","last_peered":"2024-04-25T11:42:51.598840+0000","last_clean":"2024-04-25T11:42:51.598840+0000","last_became_active":"2024-04-25T11:41:44.908747+0000","last_became_peered":"2024-04-25T11:41:44.908747+0000","last_unstale":"2024-04-25T11:42:51.598840+0000","last_undegraded":"2024-04-25T11:42:51.598840+0000","last_fullsized":"2024-04-25T11:42:51.598840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:50.281168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078658+0000","last_change":"2024-04-25T11:41:44.910692+0000","last_active":"2024-04-25T11:42:51.078658+0000","last_peered":"2024-04-25T11:42:51.078658+0000","last_clean":"2024-04-25T11:42:51.078658+0000","last_became_active":"2024-04-25T11:41:44.910343+0000","last_became_peered":"2024-04-25T11:41:44.910343+0000","last_unstale":"2024-04-25T11:42:51.078658+0000","last_undegraded":"2024-04-25T11:42:51.078658+0000","last_fullsized":"2024-04-25T11:42:51.078658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.945563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080980+0000","last_change":"2024-04-25T11:41:44.910751+0000","last_active":"2024-04-25T11:42:51.080980+0000","last_peered":"2024-04-25T11:42:51.080980+0000","last_clean":"2024-04-25T11:42:51.080980+0000","last_became_active":"2024-04-25T11:41:44.909686+0000","last_became_peered":"2024-04-25T11:41:44.909686+0000","last_unstale":"2024-04-25T11:42:51.080980+0000","last_undegraded":"2024-04-25T11:42:51.080980+0000","last_fullsized":"2024-04-25T11:42:51.080980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:37.216282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.599397+0000","last_change":"2024-04-25T11:41:44.915148+0000","last_active":"2024-04-25T11:42:51.599397+0000","last_peered":"2024-04-25T11:42:51.599397+0000","last_clean":"2024-04-25T11:42:51.599397+0000","last_became_active":"2024-04-25T11:41:44.914772+0000","last_became_peered":"2024-04-25T11:41:44.914772+0000","last_unstale":"2024-04-25T11:42:51.599397+0000","last_undegraded":"2024-04-25T11:42:51.599397+0000","last_fullsized":"2024-04-25T11:42:51.599397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:09.121386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.579956+0000","last_change":"2024-04-25T11:41:44.906569+0000","last_active":"2024-04-25T11:42:51.579956+0000","last_peered":"2024-04-25T11:42:51.579956+0000","last_clean":"2024-04-25T11:42:51.579956+0000","last_became_active":"2024-04-25T11:41:44.906259+0000","last_became_peered":"2024-04-25T11:41:44.906259+0000","last_unstale":"2024-04-25T11:42:51.579956+0000","last_undegraded":"2024-04-25T11:42:51.579956+0000","last_fullsized":"2024-04-25T11:42:51.579956+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:47.957863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.079880+0000","last_change":"2024-04-25T11:41:44.909261+0000","last_active":"2024-04-25T11:42:51.079880+0000","last_peered":"2024-04-25T11:42:51.079880+0000","last_clean":"2024-04-25T11:42:51.079880+0000","last_became_active":"2024-04-25T11:41:44.907931+0000","last_became_peered":"2024-04-25T11:41:44.907931+0000","last_unstale":"2024-04-25T11:42:51.079880+0000","last_undegraded":"2024-04-25T11:42:51.079880+0000","last_fullsized":"2024-04-25T11:42:51.079880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:35.754990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.079710+0000","last_change":"2024-04-25T11:41:44.908695+0000","last_active":"2024-04-25T11:42:51.079710+0000","last_peered":"2024-04-25T11:42:51.079710+0000","last_clean":"2024-04-25T11:42:51.079710+0000","last_became_active":"2024-04-25T11:41:44.908374+0000","last_became_peered":"2024-04-25T11:41:44.908374+0000","last_unstale":"2024-04-25T11:42:51.079710+0000","last_undegraded":"2024-04-25T11:42:51.079710+0000","last_fullsized":"2024-04-25T11:42:51.079710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:30.120631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598258+0000","last_change":"2024-04-25T11:41:44.903279+0000","last_active":"2024-04-25T11:42:51.598258+0000","last_peered":"2024-04-25T11:42:51.598258+0000","last_clean":"2024-04-25T11:42:51.598258+0000","last_became_active":"2024-04-25T11:41:44.902958+0000","last_became_peered":"2024-04-25T11:41:44.902958+0000","last_unstale":"2024-04-25T11:42:51.598258+0000","last_undegraded":"2024-04-25T11:42:51.598258+0000","last_fullsized":"2024-04-25T11:42:51.598258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:18.380774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078846+0000","last_change":"2024-04-25T11:41:44.908108+0000","last_active":"2024-04-25T11:42:51.078846+0000","last_peered":"2024-04-25T11:42:51.078846+0000","last_clean":"2024-04-25T11:42:51.078846+0000","last_became_active":"2024-04-25T11:41:44.907787+0000","last_became_peered":"2024-04-25T11:41:44.907787+0000","last_unstale":"2024-04-25T11:42:51.078846+0000","last_undegraded":"2024-04-25T11:42:51.078846+0000","last_fullsized":"2024-04-25T11:42:51.078846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:53.306845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078226+0000","last_change":"2024-04-25T11:41:44.906630+0000","last_active":"2024-04-25T11:42:51.078226+0000","last_peered":"2024-04-25T11:42:51.078226+0000","last_clean":"2024-04-25T11:42:51.078226+0000","last_became_active":"2024-04-25T11:41:44.906321+0000","last_became_peered":"2024-04-25T11:41:44.906321+0000","last_unstale":"2024-04-25T11:42:51.078226+0000","last_undegraded":"2024-04-25T11:42:51.078226+0000","last_fullsized":"2024-04-25T11:42:51.078226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:25.588918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081862+0000","last_change":"2024-04-25T11:41:44.905390+0000","last_active":"2024-04-25T11:42:51.081862+0000","last_peered":"2024-04-25T11:42:51.081862+0000","last_clean":"2024-04-25T11:42:51.081862+0000","last_became_active":"2024-04-25T11:41:44.905033+0000","last_became_peered":"2024-04-25T11:41:44.905033+0000","last_unstale":"2024-04-25T11:42:51.081862+0000","last_undegraded":"2024-04-25T11:42:51.081862+0000","last_fullsized":"2024-04-25T11:42:51.081862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:00.998434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:53.670832+0000","last_change":"2024-04-25T11:41:44.454718+0000","last_active":"2024-04-25T11:42:53.670832+0000","last_peered":"2024-04-25T11:42:53.670832+0000","last_clean":"2024-04-25T11:42:53.670832+0000","last_became_active":"2024-04-25T11:41:44.454400+0000","last_became_peered":"2024-04-25T11:41:44.454400+0000","last_unstale":"2024-04-25T11:42:53.670832+0000","last_undegraded":"2024-04-25T11:42:53.670832+0000","last_fullsized":"2024-04-25T11:42:53.670832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:05.965988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598363+0000","last_change":"2024-04-25T11:41:44.913512+0000","last_active":"2024-04-25T11:42:51.598363+0000","last_peered":"2024-04-25T11:42:51.598363+0000","last_clean":"2024-04-25T11:42:51.598363+0000","last_became_active":"2024-04-25T11:41:44.913168+0000","last_became_peered":"2024-04-25T11:41:44.913168+0000","last_unstale":"2024-04-25T11:42:51.598363+0000","last_undegraded":"2024-04-25T11:42:51.598363+0000","last_fullsized":"2024-04-25T11:42:51.598363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:25.139747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598473+0000","last_change":"2024-04-25T11:41:44.451331+0000","last_active":"2024-04-25T11:42:51.598473+0000","last_peered":"2024-04-25T11:42:51.598473+0000","last_clean":"2024-04-25T11:42:51.598473+0000","last_became_active":"2024-04-25T11:41:44.450788+0000","last_became_peered":"2024-04-25T11:41:44.450788+0000","last_unstale":"2024-04-25T11:42:51.598473+0000","last_undegraded":"2024-04-25T11:42:51.598473+0000","last_fullsized":"2024-04-25T11:42:51.598473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:49.011558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080471+0000","last_change":"2024-04-25T11:41:44.908536+0000","last_active":"2024-04-25T11:42:51.080471+0000","last_peered":"2024-04-25T11:42:51.080471+0000","last_clean":"2024-04-25T11:42:51.080471+0000","last_became_active":"2024-04-25T11:41:44.907823+0000","last_became_peered":"2024-04-25T11:41:44.907823+0000","last_unstale":"2024-04-25T11:42:51.080471+0000","last_undegraded":"2024-04-25T11:42:51.080471+0000","last_fullsized":"2024-04-25T11:42:51.080471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:19.898695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.095130+0000","last_change":"2024-04-25T11:42:51.094639+0000","last_active":"2024-04-25T11:42:51.095130+0000","last_peered":"2024-04-25T11:42:51.095130+0000","last_clean":"2024-04-25T11:42:51.095130+0000","last_became_active":"2024-04-25T11:42:51.083342+0000","last_became_peered":"2024-04-25T11:42:51.083342+0000","last_unstale":"2024-04-25T11:42:51.095130+0000","last_undegraded":"2024-04-25T11:42:51.095130+0000","last_fullsized":"2024-04-25T11:42:51.095130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:20.075324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598595+0000","last_change":"2024-04-25T11:41:44.908881+0000","last_active":"2024-04-25T11:42:51.598595+0000","last_peered":"2024-04-25T11:42:51.598595+0000","last_clean":"2024-04-25T11:42:51.598595+0000","last_became_active":"2024-04-25T11:41:44.908493+0000","last_became_peered":"2024-04-25T11:41:44.908493+0000","last_unstale":"2024-04-25T11:42:51.598595+0000","last_undegraded":"2024-04-25T11:42:51.598595+0000","last_fullsized":"2024-04-25T11:42:51.598595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:58.930420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080533+0000","last_change":"2024-04-25T11:41:44.451870+0000","last_active":"2024-04-25T11:42:51.080533+0000","last_peered":"2024-04-25T11:42:51.080533+0000","last_clean":"2024-04-25T11:42:51.080533+0000","last_became_active":"2024-04-25T11:41:44.451511+0000","last_became_peered":"2024-04-25T11:41:44.451511+0000","last_unstale":"2024-04-25T11:42:51.080533+0000","last_undegraded":"2024-04-25T11:42:51.080533+0000","last_fullsized":"2024-04-25T11:42:51.080533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:26.457919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.380169+0000","last_change":"2024-04-25T11:41:44.910738+0000","last_active":"2024-04-25T11:42:51.380169+0000","last_peered":"2024-04-25T11:42:51.380169+0000","last_clean":"2024-04-25T11:42:51.380169+0000","last_became_active":"2024-04-25T11:41:44.910415+0000","last_became_peered":"2024-04-25T11:41:44.910415+0000","last_unstale":"2024-04-25T11:42:51.380169+0000","last_undegraded":"2024-04-25T11:42:51.380169+0000","last_fullsized":"2024-04-25T11:42:51.380169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:01.333894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:57.671771+0000","last_change":"2024-04-25T11:41:44.455288+0000","last_active":"2024-04-25T11:42:57.671771+0000","last_peered":"2024-04-25T11:42:57.671771+0000","last_clean":"2024-04-25T11:42:57.671771+0000","last_became_active":"2024-04-25T11:41:44.454965+0000","last_became_peered":"2024-04-25T11:41:44.454965+0000","last_unstale":"2024-04-25T11:42:57.671771+0000","last_undegraded":"2024-04-25T11:42:57.671771+0000","last_fullsized":"2024-04-25T11:42:57.671771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:33.943287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.083170+0000","last_change":"2024-04-25T11:42:51.083170+0000","last_active":"2024-04-25T11:42:51.083170+0000","last_peered":"2024-04-25T11:42:51.083170+0000","last_clean":"2024-04-25T11:42:51.083170+0000","last_became_active":"2024-04-25T11:42:51.082750+0000","last_became_peered":"2024-04-25T11:42:51.082750+0000","last_unstale":"2024-04-25T11:42:51.083170+0000","last_undegraded":"2024-04-25T11:42:51.083170+0000","last_fullsized":"2024-04-25T11:42:51.083170+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:13.464879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078225+0000","last_change":"2024-04-25T11:41:44.451846+0000","last_active":"2024-04-25T11:42:51.078225+0000","last_peered":"2024-04-25T11:42:51.078225+0000","last_clean":"2024-04-25T11:42:51.078225+0000","last_became_active":"2024-04-25T11:41:44.451416+0000","last_became_peered":"2024-04-25T11:41:44.451416+0000","last_unstale":"2024-04-25T11:42:51.078225+0000","last_undegraded":"2024-04-25T11:42:51.078225+0000","last_fullsized":"2024-04-25T11:42:51.078225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:57.023883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.379919+0000","last_change":"2024-04-25T11:41:44.907761+0000","last_active":"2024-04-25T11:42:51.379919+0000","last_peered":"2024-04-25T11:42:51.379919+0000","last_clean":"2024-04-25T11:42:51.379919+0000","last_became_active":"2024-04-25T11:41:44.906743+0000","last_became_peered":"2024-04-25T11:41:44.906743+0000","last_unstale":"2024-04-25T11:42:51.379919+0000","last_undegraded":"2024-04-25T11:42:51.379919+0000","last_fullsized":"2024-04-25T11:42:51.379919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:43.304985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.579856+0000","last_change":"2024-04-25T11:41:44.447415+0000","last_active":"2024-04-25T11:42:51.579856+0000","last_peered":"2024-04-25T11:42:51.579856+0000","last_clean":"2024-04-25T11:42:51.579856+0000","last_became_active":"2024-04-25T11:41:44.447091+0000","last_became_peered":"2024-04-25T11:41:44.447091+0000","last_unstale":"2024-04-25T11:42:51.579856+0000","last_undegraded":"2024-04-25T11:42:51.579856+0000","last_fullsized":"2024-04-25T11:42:51.579856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:27.879115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598001+0000","last_change":"2024-04-25T11:41:44.913350+0000","last_active":"2024-04-25T11:42:51.598001+0000","last_peered":"2024-04-25T11:42:51.598001+0000","last_clean":"2024-04-25T11:42:51.598001+0000","last_became_active":"2024-04-25T11:41:44.912602+0000","last_became_peered":"2024-04-25T11:41:44.912602+0000","last_unstale":"2024-04-25T11:42:51.598001+0000","last_undegraded":"2024-04-25T11:42:51.598001+0000","last_fullsized":"2024-04-25T11:42:51.598001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:03.099520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.379589+0000","last_change":"2024-04-25T11:41:44.451283+0000","last_active":"2024-04-25T11:42:51.379589+0000","last_peered":"2024-04-25T11:42:51.379589+0000","last_clean":"2024-04-25T11:42:51.379589+0000","last_became_active":"2024-04-25T11:41:44.450955+0000","last_became_peered":"2024-04-25T11:41:44.450955+0000","last_unstale":"2024-04-25T11:42:51.379589+0000","last_undegraded":"2024-04-25T11:42:51.379589+0000","last_fullsized":"2024-04-25T11:42:51.379589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:00.912227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.077923+0000","last_change":"2024-04-25T11:41:44.904055+0000","last_active":"2024-04-25T11:42:51.077923+0000","last_peered":"2024-04-25T11:42:51.077923+0000","last_clean":"2024-04-25T11:42:51.077923+0000","last_became_active":"2024-04-25T11:41:44.903731+0000","last_became_peered":"2024-04-25T11:41:44.903731+0000","last_unstale":"2024-04-25T11:42:51.077923+0000","last_undegraded":"2024-04-25T11:42:51.077923+0000","last_fullsized":"2024-04-25T11:42:51.077923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:09.912305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":159,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:58.672611+0000","last_change":"2024-04-25T11:41:44.444989+0000","last_active":"2024-04-25T11:42:58.672611+0000","last_peered":"2024-04-25T11:42:58.672611+0000","last_clean":"2024-04-25T11:42:58.672611+0000","last_became_active":"2024-04-25T11:41:44.444543+0000","last_became_peered":"2024-04-25T11:41:44.444543+0000","last_unstale":"2024-04-25T11:42:58.672611+0000","last_undegraded":"2024-04-25T11:42:58.672611+0000","last_fullsized":"2024-04-25T11:42:58.672611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:08.957068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598644+0000","last_change":"2024-04-25T11:41:44.907435+0000","last_active":"2024-04-25T11:42:51.598644+0000","last_peered":"2024-04-25T11:42:51.598644+0000","last_clean":"2024-04-25T11:42:51.598644+0000","last_became_active":"2024-04-25T11:41:44.906972+0000","last_became_peered":"2024-04-25T11:41:44.906972+0000","last_unstale":"2024-04-25T11:42:51.598644+0000","last_undegraded":"2024-04-25T11:42:51.598644+0000","last_fullsized":"2024-04-25T11:42:51.598644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:47.011440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081068+0000","last_change":"2024-04-25T11:41:44.456804+0000","last_active":"2024-04-25T11:42:51.081068+0000","last_peered":"2024-04-25T11:42:51.081068+0000","last_clean":"2024-04-25T11:42:51.081068+0000","last_became_active":"2024-04-25T11:41:44.456472+0000","last_became_peered":"2024-04-25T11:41:44.456472+0000","last_unstale":"2024-04-25T11:42:51.081068+0000","last_undegraded":"2024-04-25T11:42:51.081068+0000","last_fullsized":"2024-04-25T11:42:51.081068+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:49.293772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.581037+0000","last_change":"2024-04-25T11:42:51.581037+0000","last_active":"2024-04-25T11:42:51.581037+0000","last_peered":"2024-04-25T11:42:51.581037+0000","last_clean":"2024-04-25T11:42:51.581037+0000","last_became_active":"2024-04-25T11:42:51.580608+0000","last_became_peered":"2024-04-25T11:42:51.580608+0000","last_unstale":"2024-04-25T11:42:51.581037+0000","last_undegraded":"2024-04-25T11:42:51.581037+0000","last_fullsized":"2024-04-25T11:42:51.581037+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:13.098088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.380226+0000","last_change":"2024-04-25T11:41:44.453860+0000","last_active":"2024-04-25T11:42:51.380226+0000","last_peered":"2024-04-25T11:42:51.380226+0000","last_clean":"2024-04-25T11:42:51.380226+0000","last_became_active":"2024-04-25T11:41:44.453536+0000","last_became_peered":"2024-04-25T11:41:44.453536+0000","last_unstale":"2024-04-25T11:42:51.380226+0000","last_undegraded":"2024-04-25T11:42:51.380226+0000","last_fullsized":"2024-04-25T11:42:51.380226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:56.762115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598982+0000","last_change":"2024-04-25T11:41:44.913443+0000","last_active":"2024-04-25T11:42:51.598982+0000","last_peered":"2024-04-25T11:42:51.598982+0000","last_clean":"2024-04-25T11:42:51.598982+0000","last_became_active":"2024-04-25T11:41:44.911350+0000","last_became_peered":"2024-04-25T11:41:44.911350+0000","last_unstale":"2024-04-25T11:42:51.598982+0000","last_undegraded":"2024-04-25T11:42:51.598982+0000","last_fullsized":"2024-04-25T11:42:51.598982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:58.345587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'27","reported_seq":186,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:58.668424+0000","last_change":"2024-04-25T11:41:44.452286+0000","last_active":"2024-04-25T11:42:58.668424+0000","last_peered":"2024-04-25T11:42:58.668424+0000","last_clean":"2024-04-25T11:42:58.668424+0000","last_became_active":"2024-04-25T11:41:44.451928+0000","last_became_peered":"2024-04-25T11:41:44.451928+0000","last_unstale":"2024-04-25T11:42:58.668424+0000","last_undegraded":"2024-04-25T11:42:58.668424+0000","last_fullsized":"2024-04-25T11:42:58.668424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:21.407375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58912,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":140,"num_read_kb":128,"num_write":29,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080880+0000","last_change":"2024-04-25T11:41:44.903909+0000","last_active":"2024-04-25T11:42:51.080880+0000","last_peered":"2024-04-25T11:42:51.080880+0000","last_clean":"2024-04-25T11:42:51.080880+0000","last_became_active":"2024-04-25T11:41:44.903620+0000","last_became_peered":"2024-04-25T11:41:44.903620+0000","last_unstale":"2024-04-25T11:42:51.080880+0000","last_undegraded":"2024-04-25T11:42:51.080880+0000","last_fullsized":"2024-04-25T11:42:51.080880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:49.910052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081746+0000","last_change":"2024-04-25T11:41:44.454294+0000","last_active":"2024-04-25T11:42:51.081746+0000","last_peered":"2024-04-25T11:42:51.081746+0000","last_clean":"2024-04-25T11:42:51.081746+0000","last_became_active":"2024-04-25T11:41:44.453829+0000","last_became_peered":"2024-04-25T11:41:44.453829+0000","last_unstale":"2024-04-25T11:42:51.081746+0000","last_undegraded":"2024-04-25T11:42:51.081746+0000","last_fullsized":"2024-04-25T11:42:51.081746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:14.617963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.579841+0000","last_change":"2024-04-25T11:41:44.907453+0000","last_active":"2024-04-25T11:42:51.579841+0000","last_peered":"2024-04-25T11:42:51.579841+0000","last_clean":"2024-04-25T11:42:51.579841+0000","last_became_active":"2024-04-25T11:41:44.907126+0000","last_became_peered":"2024-04-25T11:41:44.907126+0000","last_unstale":"2024-04-25T11:42:51.579841+0000","last_undegraded":"2024-04-25T11:42:51.579841+0000","last_fullsized":"2024-04-25T11:42:51.579841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:25.998686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.379835+0000","last_change":"2024-04-25T11:41:44.452258+0000","last_active":"2024-04-25T11:42:51.379835+0000","last_peered":"2024-04-25T11:42:51.379835+0000","last_clean":"2024-04-25T11:42:51.379835+0000","last_became_active":"2024-04-25T11:41:44.451880+0000","last_became_peered":"2024-04-25T11:41:44.451880+0000","last_unstale":"2024-04-25T11:42:51.379835+0000","last_undegraded":"2024-04-25T11:42:51.379835+0000","last_fullsized":"2024-04-25T11:42:51.379835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:39.463406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080784+0000","last_change":"2024-04-25T11:41:44.913841+0000","last_active":"2024-04-25T11:42:51.080784+0000","last_peered":"2024-04-25T11:42:51.080784+0000","last_clean":"2024-04-25T11:42:51.080784+0000","last_became_active":"2024-04-25T11:41:44.913378+0000","last_became_peered":"2024-04-25T11:41:44.913378+0000","last_unstale":"2024-04-25T11:42:51.080784+0000","last_undegraded":"2024-04-25T11:42:51.080784+0000","last_fullsized":"2024-04-25T11:42:51.080784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:24.732899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:53.671833+0000","last_change":"2024-04-25T11:41:44.450531+0000","last_active":"2024-04-25T11:42:53.671833+0000","last_peered":"2024-04-25T11:42:53.671833+0000","last_clean":"2024-04-25T11:42:53.671833+0000","last_became_active":"2024-04-25T11:41:44.450205+0000","last_became_peered":"2024-04-25T11:41:44.450205+0000","last_unstale":"2024-04-25T11:42:53.671833+0000","last_undegraded":"2024-04-25T11:42:53.671833+0000","last_fullsized":"2024-04-25T11:42:53.671833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:51.797969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080312+0000","last_change":"2024-04-25T11:41:44.911883+0000","last_active":"2024-04-25T11:42:51.080312+0000","last_peered":"2024-04-25T11:42:51.080312+0000","last_clean":"2024-04-25T11:42:51.080312+0000","last_became_active":"2024-04-25T11:41:44.911362+0000","last_became_peered":"2024-04-25T11:41:44.911362+0000","last_unstale":"2024-04-25T11:42:51.080312+0000","last_undegraded":"2024-04-25T11:42:51.080312+0000","last_fullsized":"2024-04-25T11:42:51.080312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:34.273876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080169+0000","last_change":"2024-04-25T11:41:44.455844+0000","last_active":"2024-04-25T11:42:51.080169+0000","last_peered":"2024-04-25T11:42:51.080169+0000","last_clean":"2024-04-25T11:42:51.080169+0000","last_became_active":"2024-04-25T11:41:44.455167+0000","last_became_peered":"2024-04-25T11:41:44.455167+0000","last_unstale":"2024-04-25T11:42:51.080169+0000","last_undegraded":"2024-04-25T11:42:51.080169+0000","last_fullsized":"2024-04-25T11:42:51.080169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.886078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.599983+0000","last_change":"2024-04-25T11:42:51.599983+0000","last_active":"2024-04-25T11:42:51.599983+0000","last_peered":"2024-04-25T11:42:51.599983+0000","last_clean":"2024-04-25T11:42:51.599983+0000","last_became_active":"2024-04-25T11:42:51.599536+0000","last_became_peered":"2024-04-25T11:42:51.599536+0000","last_unstale":"2024-04-25T11:42:51.599983+0000","last_undegraded":"2024-04-25T11:42:51.599983+0000","last_fullsized":"2024-04-25T11:42:51.599983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:46.705406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080962+0000","last_change":"2024-04-25T11:41:44.452371+0000","last_active":"2024-04-25T11:42:51.080962+0000","last_peered":"2024-04-25T11:42:51.080962+0000","last_clean":"2024-04-25T11:42:51.080962+0000","last_became_active":"2024-04-25T11:41:44.451406+0000","last_became_peered":"2024-04-25T11:41:44.451406+0000","last_unstale":"2024-04-25T11:42:51.080962+0000","last_undegraded":"2024-04-25T11:42:51.080962+0000","last_fullsized":"2024-04-25T11:42:51.080962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:00.093835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078136+0000","last_change":"2024-04-25T11:41:44.910999+0000","last_active":"2024-04-25T11:42:51.078136+0000","last_peered":"2024-04-25T11:42:51.078136+0000","last_clean":"2024-04-25T11:42:51.078136+0000","last_became_active":"2024-04-25T11:41:44.909749+0000","last_became_peered":"2024-04-25T11:41:44.909749+0000","last_unstale":"2024-04-25T11:42:51.078136+0000","last_undegraded":"2024-04-25T11:42:51.078136+0000","last_fullsized":"2024-04-25T11:42:51.078136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:44.272145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.083142+0000","last_change":"2024-04-25T11:42:51.083142+0000","last_active":"2024-04-25T11:42:51.083142+0000","last_peered":"2024-04-25T11:42:51.083142+0000","last_clean":"2024-04-25T11:42:51.083142+0000","last_became_active":"2024-04-25T11:42:51.082717+0000","last_became_peered":"2024-04-25T11:42:51.082717+0000","last_unstale":"2024-04-25T11:42:51.083142+0000","last_undegraded":"2024-04-25T11:42:51.083142+0000","last_fullsized":"2024-04-25T11:42:51.083142+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:11.639907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078611+0000","last_change":"2024-04-25T11:41:44.907350+0000","last_active":"2024-04-25T11:42:51.078611+0000","last_peered":"2024-04-25T11:42:51.078611+0000","last_clean":"2024-04-25T11:42:51.078611+0000","last_became_active":"2024-04-25T11:41:44.907000+0000","last_became_peered":"2024-04-25T11:41:44.907000+0000","last_unstale":"2024-04-25T11:42:51.078611+0000","last_undegraded":"2024-04-25T11:42:51.078611+0000","last_fullsized":"2024-04-25T11:42:51.078611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:08.823550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:58.677936+0000","last_change":"2024-04-25T11:41:44.455318+0000","last_active":"2024-04-25T11:42:58.677936+0000","last_peered":"2024-04-25T11:42:58.677936+0000","last_clean":"2024-04-25T11:42:58.677936+0000","last_became_active":"2024-04-25T11:41:44.454972+0000","last_became_peered":"2024-04-25T11:41:44.454972+0000","last_unstale":"2024-04-25T11:42:58.677936+0000","last_undegraded":"2024-04-25T11:42:58.677936+0000","last_fullsized":"2024-04-25T11:42:58.677936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.255860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078641+0000","last_change":"2024-04-25T11:41:44.912507+0000","last_active":"2024-04-25T11:42:51.078641+0000","last_peered":"2024-04-25T11:42:51.078641+0000","last_clean":"2024-04-25T11:42:51.078641+0000","last_became_active":"2024-04-25T11:41:44.912123+0000","last_became_peered":"2024-04-25T11:41:44.912123+0000","last_unstale":"2024-04-25T11:42:51.078641+0000","last_undegraded":"2024-04-25T11:42:51.078641+0000","last_fullsized":"2024-04-25T11:42:51.078641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:40.659961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.379732+0000","last_change":"2024-04-25T11:41:44.913505+0000","last_active":"2024-04-25T11:42:51.379732+0000","last_peered":"2024-04-25T11:42:51.379732+0000","last_clean":"2024-04-25T11:42:51.379732+0000","last_became_active":"2024-04-25T11:41:44.913103+0000","last_became_peered":"2024-04-25T11:41:44.913103+0000","last_unstale":"2024-04-25T11:42:51.379732+0000","last_undegraded":"2024-04-25T11:42:51.379732+0000","last_fullsized":"2024-04-25T11:42:51.379732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:49.414802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:58.678112+0000","last_change":"2024-04-25T11:41:44.450056+0000","last_active":"2024-04-25T11:42:58.678112+0000","last_peered":"2024-04-25T11:42:58.678112+0000","last_clean":"2024-04-25T11:42:58.678112+0000","last_became_active":"2024-04-25T11:41:44.449754+0000","last_became_peered":"2024-04-25T11:41:44.449754+0000","last_unstale":"2024-04-25T11:42:58.678112+0000","last_undegraded":"2024-04-25T11:42:58.678112+0000","last_fullsized":"2024-04-25T11:42:58.678112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:30.059852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598221+0000","last_change":"2024-04-25T11:41:44.915061+0000","last_active":"2024-04-25T11:42:51.598221+0000","last_peered":"2024-04-25T11:42:51.598221+0000","last_clean":"2024-04-25T11:42:51.598221+0000","last_became_active":"2024-04-25T11:41:44.914799+0000","last_became_peered":"2024-04-25T11:41:44.914799+0000","last_unstale":"2024-04-25T11:42:51.598221+0000","last_undegraded":"2024-04-25T11:42:51.598221+0000","last_fullsized":"2024-04-25T11:42:51.598221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:22.820539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.599445+0000","last_change":"2024-04-25T11:41:44.449248+0000","last_active":"2024-04-25T11:42:51.599445+0000","last_peered":"2024-04-25T11:42:51.599445+0000","last_clean":"2024-04-25T11:42:51.599445+0000","last_became_active":"2024-04-25T11:41:44.448898+0000","last_became_peered":"2024-04-25T11:41:44.448898+0000","last_unstale":"2024-04-25T11:42:51.599445+0000","last_undegraded":"2024-04-25T11:42:51.599445+0000","last_fullsized":"2024-04-25T11:42:51.599445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:19.574270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080486+0000","last_change":"2024-04-25T11:40:36.892561+0000","last_active":"2024-04-25T11:42:51.080486+0000","last_peered":"2024-04-25T11:42:51.080486+0000","last_clean":"2024-04-25T11:42:51.080486+0000","last_became_active":"2024-04-25T11:40:36.892126+0000","last_became_peered":"2024-04-25T11:40:36.892126+0000","last_unstale":"2024-04-25T11:42:51.080486+0000","last_undegraded":"2024-04-25T11:42:51.080486+0000","last_fullsized":"2024-04-25T11:42:51.080486+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:37:16.666579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:37:16.666579+0000","last_clean_scrub_stamp":"2024-04-25T11:37:16.666579+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:44.270847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.579703+0000","last_change":"2024-04-25T11:41:44.908446+0000","last_active":"2024-04-25T11:42:51.579703+0000","last_peered":"2024-04-25T11:42:51.579703+0000","last_clean":"2024-04-25T11:42:51.579703+0000","last_became_active":"2024-04-25T11:41:44.908148+0000","last_became_peered":"2024-04-25T11:41:44.908148+0000","last_unstale":"2024-04-25T11:42:51.579703+0000","last_undegraded":"2024-04-25T11:42:51.579703+0000","last_fullsized":"2024-04-25T11:42:51.579703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:09.598976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080643+0000","last_change":"2024-04-25T11:41:44.454815+0000","last_active":"2024-04-25T11:42:51.080643+0000","last_peered":"2024-04-25T11:42:51.080643+0000","last_clean":"2024-04-25T11:42:51.080643+0000","last_became_active":"2024-04-25T11:41:44.454521+0000","last_became_peered":"2024-04-25T11:41:44.454521+0000","last_unstale":"2024-04-25T11:42:51.080643+0000","last_undegraded":"2024-04-25T11:42:51.080643+0000","last_fullsized":"2024-04-25T11:42:51.080643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:45.932329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081011+0000","last_change":"2024-04-25T11:41:44.456641+0000","last_active":"2024-04-25T11:42:51.081011+0000","last_peered":"2024-04-25T11:42:51.081011+0000","last_clean":"2024-04-25T11:42:51.081011+0000","last_became_active":"2024-04-25T11:41:44.456133+0000","last_became_peered":"2024-04-25T11:41:44.456133+0000","last_unstale":"2024-04-25T11:42:51.081011+0000","last_undegraded":"2024-04-25T11:42:51.081011+0000","last_fullsized":"2024-04-25T11:42:51.081011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:47.746160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.580086+0000","last_change":"2024-04-25T11:41:44.897512+0000","last_active":"2024-04-25T11:42:51.580086+0000","last_peered":"2024-04-25T11:42:51.580086+0000","last_clean":"2024-04-25T11:42:51.580086+0000","last_became_active":"2024-04-25T11:41:44.897184+0000","last_became_peered":"2024-04-25T11:41:44.897184+0000","last_unstale":"2024-04-25T11:42:51.580086+0000","last_undegraded":"2024-04-25T11:42:51.580086+0000","last_fullsized":"2024-04-25T11:42:51.580086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:53.703825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078338+0000","last_change":"2024-04-25T11:41:44.458318+0000","last_active":"2024-04-25T11:42:51.078338+0000","last_peered":"2024-04-25T11:42:51.078338+0000","last_clean":"2024-04-25T11:42:51.078338+0000","last_became_active":"2024-04-25T11:41:44.458001+0000","last_became_peered":"2024-04-25T11:41:44.458001+0000","last_unstale":"2024-04-25T11:42:51.078338+0000","last_undegraded":"2024-04-25T11:42:51.078338+0000","last_fullsized":"2024-04-25T11:42:51.078338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:17.100621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081876+0000","last_change":"2024-04-25T11:42:51.081876+0000","last_active":"2024-04-25T11:42:51.081876+0000","last_peered":"2024-04-25T11:42:51.081876+0000","last_clean":"2024-04-25T11:42:51.081876+0000","last_became_active":"2024-04-25T11:42:51.081566+0000","last_became_peered":"2024-04-25T11:42:51.081566+0000","last_unstale":"2024-04-25T11:42:51.081876+0000","last_undegraded":"2024-04-25T11:42:51.081876+0000","last_fullsized":"2024-04-25T11:42:51.081876+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:05.530080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.599402+0000","last_change":"2024-04-25T11:41:44.460492+0000","last_active":"2024-04-25T11:42:51.599402+0000","last_peered":"2024-04-25T11:42:51.599402+0000","last_clean":"2024-04-25T11:42:51.599402+0000","last_became_active":"2024-04-25T11:41:44.460199+0000","last_became_peered":"2024-04-25T11:41:44.460199+0000","last_unstale":"2024-04-25T11:42:51.599402+0000","last_undegraded":"2024-04-25T11:42:51.599402+0000","last_fullsized":"2024-04-25T11:42:51.599402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:18.756779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080394+0000","last_change":"2024-04-25T11:41:44.909438+0000","last_active":"2024-04-25T11:42:51.080394+0000","last_peered":"2024-04-25T11:42:51.080394+0000","last_clean":"2024-04-25T11:42:51.080394+0000","last_became_active":"2024-04-25T11:41:44.909146+0000","last_became_peered":"2024-04-25T11:41:44.909146+0000","last_unstale":"2024-04-25T11:42:51.080394+0000","last_undegraded":"2024-04-25T11:42:51.080394+0000","last_fullsized":"2024-04-25T11:42:51.080394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:15.949652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598415+0000","last_change":"2024-04-25T11:41:44.453169+0000","last_active":"2024-04-25T11:42:51.598415+0000","last_peered":"2024-04-25T11:42:51.598415+0000","last_clean":"2024-04-25T11:42:51.598415+0000","last_became_active":"2024-04-25T11:41:44.452762+0000","last_became_peered":"2024-04-25T11:41:44.452762+0000","last_unstale":"2024-04-25T11:42:51.598415+0000","last_undegraded":"2024-04-25T11:42:51.598415+0000","last_fullsized":"2024-04-25T11:42:51.598415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:13.084197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.079807+0000","last_change":"2024-04-25T11:41:44.910810+0000","last_active":"2024-04-25T11:42:51.079807+0000","last_peered":"2024-04-25T11:42:51.079807+0000","last_clean":"2024-04-25T11:42:51.079807+0000","last_became_active":"2024-04-25T11:41:44.910488+0000","last_became_peered":"2024-04-25T11:41:44.910488+0000","last_unstale":"2024-04-25T11:42:51.079807+0000","last_undegraded":"2024-04-25T11:42:51.079807+0000","last_fullsized":"2024-04-25T11:42:51.079807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:36.971289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598001+0000","last_change":"2024-04-25T11:41:44.455402+0000","last_active":"2024-04-25T11:42:51.598001+0000","last_peered":"2024-04-25T11:42:51.598001+0000","last_clean":"2024-04-25T11:42:51.598001+0000","last_became_active":"2024-04-25T11:41:44.455057+0000","last_became_peered":"2024-04-25T11:41:44.455057+0000","last_unstale":"2024-04-25T11:42:51.598001+0000","last_undegraded":"2024-04-25T11:42:51.598001+0000","last_fullsized":"2024-04-25T11:42:51.598001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:32.156664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.077934+0000","last_change":"2024-04-25T11:41:44.903907+0000","last_active":"2024-04-25T11:42:51.077934+0000","last_peered":"2024-04-25T11:42:51.077934+0000","last_clean":"2024-04-25T11:42:51.077934+0000","last_became_active":"2024-04-25T11:41:44.903510+0000","last_became_peered":"2024-04-25T11:41:44.903510+0000","last_unstale":"2024-04-25T11:42:51.077934+0000","last_undegraded":"2024-04-25T11:42:51.077934+0000","last_fullsized":"2024-04-25T11:42:51.077934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:33.183412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081108+0000","last_change":"2024-04-25T11:41:44.454049+0000","last_active":"2024-04-25T11:42:51.081108+0000","last_peered":"2024-04-25T11:42:51.081108+0000","last_clean":"2024-04-25T11:42:51.081108+0000","last_became_active":"2024-04-25T11:41:44.453652+0000","last_became_peered":"2024-04-25T11:41:44.453652+0000","last_unstale":"2024-04-25T11:42:51.081108+0000","last_undegraded":"2024-04-25T11:42:51.081108+0000","last_fullsized":"2024-04-25T11:42:51.081108+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:17.239715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598209+0000","last_change":"2024-04-25T11:41:44.915005+0000","last_active":"2024-04-25T11:42:51.598209+0000","last_peered":"2024-04-25T11:42:51.598209+0000","last_clean":"2024-04-25T11:42:51.598209+0000","last_became_active":"2024-04-25T11:41:44.914663+0000","last_became_peered":"2024-04-25T11:41:44.914663+0000","last_unstale":"2024-04-25T11:42:51.598209+0000","last_undegraded":"2024-04-25T11:42:51.598209+0000","last_fullsized":"2024-04-25T11:42:51.598209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:49.390846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080022+0000","last_change":"2024-04-25T11:41:44.454194+0000","last_active":"2024-04-25T11:42:51.080022+0000","last_peered":"2024-04-25T11:42:51.080022+0000","last_clean":"2024-04-25T11:42:51.080022+0000","last_became_active":"2024-04-25T11:41:44.453280+0000","last_became_peered":"2024-04-25T11:41:44.453280+0000","last_unstale":"2024-04-25T11:42:51.080022+0000","last_undegraded":"2024-04-25T11:42:51.080022+0000","last_fullsized":"2024-04-25T11:42:51.080022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:45.368132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080654+0000","last_change":"2024-04-25T11:41:44.907951+0000","last_active":"2024-04-25T11:42:51.080654+0000","last_peered":"2024-04-25T11:42:51.080654+0000","last_clean":"2024-04-25T11:42:51.080654+0000","last_became_active":"2024-04-25T11:41:44.907554+0000","last_became_peered":"2024-04-25T11:41:44.907554+0000","last_unstale":"2024-04-25T11:42:51.080654+0000","last_undegraded":"2024-04-25T11:42:51.080654+0000","last_fullsized":"2024-04-25T11:42:51.080654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:33.312265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:55.668728+0000","last_change":"2024-04-25T11:41:44.458803+0000","last_active":"2024-04-25T11:42:55.668728+0000","last_peered":"2024-04-25T11:42:55.668728+0000","last_clean":"2024-04-25T11:42:55.668728+0000","last_became_active":"2024-04-25T11:41:44.458307+0000","last_became_peered":"2024-04-25T11:41:44.458307+0000","last_unstale":"2024-04-25T11:42:55.668728+0000","last_undegraded":"2024-04-25T11:42:55.668728+0000","last_fullsized":"2024-04-25T11:42:55.668728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:26.186429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081013+0000","last_change":"2024-04-25T11:41:44.909600+0000","last_active":"2024-04-25T11:42:51.081013+0000","last_peered":"2024-04-25T11:42:51.081013+0000","last_clean":"2024-04-25T11:42:51.081013+0000","last_became_active":"2024-04-25T11:41:44.909083+0000","last_became_peered":"2024-04-25T11:41:44.909083+0000","last_unstale":"2024-04-25T11:42:51.081013+0000","last_undegraded":"2024-04-25T11:42:51.081013+0000","last_fullsized":"2024-04-25T11:42:51.081013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:53.854631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.380077+0000","last_change":"2024-04-25T11:41:44.451466+0000","last_active":"2024-04-25T11:42:51.380077+0000","last_peered":"2024-04-25T11:42:51.380077+0000","last_clean":"2024-04-25T11:42:51.380077+0000","last_became_active":"2024-04-25T11:41:44.450866+0000","last_became_peered":"2024-04-25T11:41:44.450866+0000","last_unstale":"2024-04-25T11:42:51.380077+0000","last_undegraded":"2024-04-25T11:42:51.380077+0000","last_fullsized":"2024-04-25T11:42:51.380077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:03.715468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080753+0000","last_change":"2024-04-25T11:41:44.909789+0000","last_active":"2024-04-25T11:42:51.080753+0000","last_peered":"2024-04-25T11:42:51.080753+0000","last_clean":"2024-04-25T11:42:51.080753+0000","last_became_active":"2024-04-25T11:41:44.909490+0000","last_became_peered":"2024-04-25T11:41:44.909490+0000","last_unstale":"2024-04-25T11:42:51.080753+0000","last_undegraded":"2024-04-25T11:42:51.080753+0000","last_fullsized":"2024-04-25T11:42:51.080753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:50.543847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.580097+0000","last_change":"2024-04-25T11:41:44.449371+0000","last_active":"2024-04-25T11:42:51.580097+0000","last_peered":"2024-04-25T11:42:51.580097+0000","last_clean":"2024-04-25T11:42:51.580097+0000","last_became_active":"2024-04-25T11:41:44.448927+0000","last_became_peered":"2024-04-25T11:41:44.448927+0000","last_unstale":"2024-04-25T11:42:51.580097+0000","last_undegraded":"2024-04-25T11:42:51.580097+0000","last_fullsized":"2024-04-25T11:42:51.580097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:43.425380+0000","last_clean_scrub_stamp":"2024-04-25T11:41:43.425380+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:49.581010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081092+0000","last_change":"2024-04-25T11:41:44.906540+0000","last_active":"2024-04-25T11:42:51.081092+0000","last_peered":"2024-04-25T11:42:51.081092+0000","last_clean":"2024-04-25T11:42:51.081092+0000","last_became_active":"2024-04-25T11:41:44.905918+0000","last_became_peered":"2024-04-25T11:41:44.905918+0000","last_unstale":"2024-04-25T11:42:51.081092+0000","last_undegraded":"2024-04-25T11:42:51.081092+0000","last_fullsized":"2024-04-25T11:42:51.081092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:38.350461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078014+0000","last_change":"2024-04-25T11:41:44.904656+0000","last_active":"2024-04-25T11:42:51.078014+0000","last_peered":"2024-04-25T11:42:51.078014+0000","last_clean":"2024-04-25T11:42:51.078014+0000","last_became_active":"2024-04-25T11:41:44.904322+0000","last_became_peered":"2024-04-25T11:41:44.904322+0000","last_unstale":"2024-04-25T11:42:51.078014+0000","last_undegraded":"2024-04-25T11:42:51.078014+0000","last_fullsized":"2024-04-25T11:42:51.078014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:07.762816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081934+0000","last_change":"2024-04-25T11:41:44.911662+0000","last_active":"2024-04-25T11:42:51.081934+0000","last_peered":"2024-04-25T11:42:51.081934+0000","last_clean":"2024-04-25T11:42:51.081934+0000","last_became_active":"2024-04-25T11:41:44.911295+0000","last_became_peered":"2024-04-25T11:41:44.911295+0000","last_unstale":"2024-04-25T11:42:51.081934+0000","last_undegraded":"2024-04-25T11:42:51.081934+0000","last_fullsized":"2024-04-25T11:42:51.081934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:12.988118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.080707+0000","last_change":"2024-04-25T11:41:44.906303+0000","last_active":"2024-04-25T11:42:51.080707+0000","last_peered":"2024-04-25T11:42:51.080707+0000","last_clean":"2024-04-25T11:42:51.080707+0000","last_became_active":"2024-04-25T11:41:44.905951+0000","last_became_peered":"2024-04-25T11:41:44.905951+0000","last_unstale":"2024-04-25T11:42:51.080707+0000","last_undegraded":"2024-04-25T11:42:51.080707+0000","last_fullsized":"2024-04-25T11:42:51.080707+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:49.887856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.580929+0000","last_change":"2024-04-25T11:42:51.580929+0000","last_active":"2024-04-25T11:42:51.580929+0000","last_peered":"2024-04-25T11:42:51.580929+0000","last_clean":"2024-04-25T11:42:51.580929+0000","last_became_active":"2024-04-25T11:42:51.580492+0000","last_became_peered":"2024-04-25T11:42:51.580492+0000","last_unstale":"2024-04-25T11:42:51.580929+0000","last_undegraded":"2024-04-25T11:42:51.580929+0000","last_fullsized":"2024-04-25T11:42:51.580929+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:16.272379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.579960+0000","last_change":"2024-04-25T11:41:44.909391+0000","last_active":"2024-04-25T11:42:51.579960+0000","last_peered":"2024-04-25T11:42:51.579960+0000","last_clean":"2024-04-25T11:42:51.579960+0000","last_became_active":"2024-04-25T11:41:44.909088+0000","last_became_peered":"2024-04-25T11:41:44.909088+0000","last_unstale":"2024-04-25T11:42:51.579960+0000","last_undegraded":"2024-04-25T11:42:51.579960+0000","last_fullsized":"2024-04-25T11:42:51.579960+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:38.550716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.078275+0000","last_change":"2024-04-25T11:41:44.904351+0000","last_active":"2024-04-25T11:42:51.078275+0000","last_peered":"2024-04-25T11:42:51.078275+0000","last_clean":"2024-04-25T11:42:51.078275+0000","last_became_active":"2024-04-25T11:41:44.904015+0000","last_became_peered":"2024-04-25T11:41:44.904015+0000","last_unstale":"2024-04-25T11:42:51.078275+0000","last_undegraded":"2024-04-25T11:42:51.078275+0000","last_fullsized":"2024-04-25T11:42:51.078275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:18.382370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.081641+0000","last_change":"2024-04-25T11:41:44.901469+0000","last_active":"2024-04-25T11:42:51.081641+0000","last_peered":"2024-04-25T11:42:51.081641+0000","last_clean":"2024-04-25T11:42:51.081641+0000","last_became_active":"2024-04-25T11:41:44.900943+0000","last_became_peered":"2024-04-25T11:41:44.900943+0000","last_unstale":"2024-04-25T11:42:51.081641+0000","last_undegraded":"2024-04-25T11:42:51.081641+0000","last_fullsized":"2024-04-25T11:42:51.081641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:33.712132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598326+0000","last_change":"2024-04-25T11:41:44.912267+0000","last_active":"2024-04-25T11:42:51.598326+0000","last_peered":"2024-04-25T11:42:51.598326+0000","last_clean":"2024-04-25T11:42:51.598326+0000","last_became_active":"2024-04-25T11:41:44.911968+0000","last_became_peered":"2024-04-25T11:41:44.911968+0000","last_unstale":"2024-04-25T11:42:51.598326+0000","last_undegraded":"2024-04-25T11:42:51.598326+0000","last_fullsized":"2024-04-25T11:42:51.598326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:27.956046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.379651+0000","last_change":"2024-04-25T11:41:44.907540+0000","last_active":"2024-04-25T11:42:51.379651+0000","last_peered":"2024-04-25T11:42:51.379651+0000","last_clean":"2024-04-25T11:42:51.379651+0000","last_became_active":"2024-04-25T11:41:44.907229+0000","last_became_peered":"2024-04-25T11:41:44.907229+0000","last_unstale":"2024-04-25T11:42:51.379651+0000","last_undegraded":"2024-04-25T11:42:51.379651+0000","last_fullsized":"2024-04-25T11:42:51.379651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:49.123196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.597941+0000","last_change":"2024-04-25T11:41:44.912354+0000","last_active":"2024-04-25T11:42:51.597941+0000","last_peered":"2024-04-25T11:42:51.597941+0000","last_clean":"2024-04-25T11:42:51.597941+0000","last_became_active":"2024-04-25T11:41:44.912028+0000","last_became_peered":"2024-04-25T11:41:44.912028+0000","last_unstale":"2024-04-25T11:42:51.597941+0000","last_undegraded":"2024-04-25T11:42:51.597941+0000","last_fullsized":"2024-04-25T11:42:51.597941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:50.824647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.598906+0000","last_change":"2024-04-25T11:41:44.907346+0000","last_active":"2024-04-25T11:42:51.598906+0000","last_peered":"2024-04-25T11:42:51.598906+0000","last_clean":"2024-04-25T11:42:51.598906+0000","last_became_active":"2024-04-25T11:41:44.907027+0000","last_became_peered":"2024-04-25T11:41:44.907027+0000","last_unstale":"2024-04-25T11:42:51.598906+0000","last_undegraded":"2024-04-25T11:42:51.598906+0000","last_fullsized":"2024-04-25T11:42:51.598906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:01.913218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.580008+0000","last_change":"2024-04-25T11:41:44.902606+0000","last_active":"2024-04-25T11:42:51.580008+0000","last_peered":"2024-04-25T11:42:51.580008+0000","last_clean":"2024-04-25T11:42:51.580008+0000","last_became_active":"2024-04-25T11:41:44.902306+0000","last_became_peered":"2024-04-25T11:41:44.902306+0000","last_unstale":"2024-04-25T11:42:51.580008+0000","last_undegraded":"2024-04-25T11:42:51.580008+0000","last_fullsized":"2024-04-25T11:42:51.580008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:24.214013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T11:42:51.083688+0000","last_change":"2024-04-25T11:42:51.083688+0000","last_active":"2024-04-25T11:42:51.083688+0000","last_peered":"2024-04-25T11:42:51.083688+0000","last_clean":"2024-04-25T11:42:51.083688+0000","last_became_active":"2024-04-25T11:42:51.082806+0000","last_became_peered":"2024-04-25T11:42:51.082806+0000","last_unstale":"2024-04-25T11:42:51.083688+0000","last_undegraded":"2024-04-25T11:42:51.083688+0000","last_fullsized":"2024-04-25T11:42:51.083688+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:41:44.431458+0000","last_clean_scrub_stamp":"2024-04-25T11:41:44.431458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:03.684385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66116,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":704,"num_read_kb":425,"num_write":234,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":198348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351003,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":460429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1658880,"data_stored":1110761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776166,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":460359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037802,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":460737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299440,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708268,"statfs":{"total":95995035648,"available":95957266432,"internally_reserved":0,"allocated":1576960,"data_stored":1053646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":966656,"data_stored":460076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":519901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":461415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":520512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":459901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888126,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":460328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724601,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1589248,"data_stored":1052933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":60664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:43:02.332 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T11:43:02.332 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T11:43:02.332 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T11:43:02.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T11:43:02.578 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:43:03.225 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:02 smithi094 ceph-mon[33032]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:43:03.225 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:02 smithi094 ceph-mon[33032]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T11:43:03.274 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:02 smithi111 ceph-mon[32784]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:43:03.274 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:02 smithi111 ceph-mon[32784]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T11:43:03.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:02 smithi083 ceph-mon[29330]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T11:43:03.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:02 smithi083 ceph-mon[29330]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T11:43:04.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.115+0000 7f285ac3f700 1 -- 172.21.15.83:0/3494409334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2854102a90 msgr2=0x7f2854102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:04.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.115+0000 7f285ac3f700 1 --2- 172.21.15.83:0/3494409334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2854102a90 0x7f2854102f10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2840009990 tx=0x7f2840009ca0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.115+0000 7f285ac3f700 1 -- 172.21.15.83:0/3494409334 shutdown_connections 2024-04-25T11:43:04.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.115+0000 7f285ac3f700 1 --2- 172.21.15.83:0/3494409334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2854103450 0x7f285410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.115+0000 7f285ac3f700 1 --2- 172.21.15.83:0/3494409334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2854108a50 0x7f2854108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.115+0000 7f285ac3f700 1 --2- 172.21.15.83:0/3494409334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2854102a90 0x7f2854102f10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.115+0000 7f285ac3f700 1 -- 172.21.15.83:0/3494409334 >> 172.21.15.83:0/3494409334 conn(0x7f2854079ba0 msgr2=0x7f2854079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.115+0000 7f285ac3f700 1 -- 172.21.15.83:0/3494409334 shutdown_connections 2024-04-25T11:43:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.115+0000 7f285ac3f700 1 -- 172.21.15.83:0/3494409334 wait complete. 2024-04-25T11:43:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.115+0000 7f285ac3f700 1 Processor -- start 2024-04-25T11:43:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.115+0000 7f285ac3f700 1 -- start start 2024-04-25T11:43:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f285ac3f700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2854102a90 0x7f2854199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f285ac3f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2854103450 0x7f2854199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f285ac3f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2854108a50 0x7f285419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f285ac3f700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2854112cd0 con 0x7f2854102a90 2024-04-25T11:43:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f285ac3f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2854112b50 con 0x7f2854108a50 2024-04-25T11:43:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f285ac3f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2854112e50 con 0x7f2854103450 2024-04-25T11:43:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f28589db700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2854102a90 0x7f2854199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f28589db700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2854102a90 0x7f2854199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60832/0 (socket says 172.21.15.83:60832) 2024-04-25T11:43:04.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f28589db700 1 -- 172.21.15.83:0/878673478 learned_addr learned my addr 172.21.15.83:0/878673478 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:04.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f2853fff700 1 --2- 172.21.15.83:0/878673478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2854103450 0x7f2854199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f28589db700 1 -- 172.21.15.83:0/878673478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2854103450 msgr2=0x7f2854199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:04.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f28589db700 1 --2- 172.21.15.83:0/878673478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2854103450 0x7f2854199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f28589db700 1 -- 172.21.15.83:0/878673478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2854108a50 msgr2=0x7f285419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:04.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f28591dc700 1 --2- 172.21.15.83:0/878673478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2854108a50 0x7f285419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f28589db700 1 --2- 172.21.15.83:0/878673478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2854108a50 0x7f285419dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f28589db700 1 -- 172.21.15.83:0/878673478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285419e210 con 0x7f2854102a90 2024-04-25T11:43:04.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f2853fff700 1 --2- 172.21.15.83:0/878673478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2854103450 0x7f2854199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:04.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f28589db700 1 --2- 172.21.15.83:0/878673478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2854102a90 0x7f2854199200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f284800d980 tx=0x7f284800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:04.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f2851ffb700 1 -- 172.21.15.83:0/878673478 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848017070 con 0x7f2854102a90 2024-04-25T11:43:04.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f285ac3f700 1 -- 172.21.15.83:0/878673478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285419e500 con 0x7f2854102a90 2024-04-25T11:43:04.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f2851ffb700 1 -- 172.21.15.83:0/878673478 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2848004830 con 0x7f2854102a90 2024-04-25T11:43:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.116+0000 7f2851ffb700 1 -- 172.21.15.83:0/878673478 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848005440 con 0x7f2854102a90 2024-04-25T11:43:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.117+0000 7f285ac3f700 1 -- 172.21.15.83:0/878673478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28541a5d80 con 0x7f2854102a90 2024-04-25T11:43:04.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.117+0000 7f285ac3f700 1 -- 172.21.15.83:0/878673478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2854069730 con 0x7f2854102a90 2024-04-25T11:43:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.118+0000 7f2851ffb700 1 -- 172.21.15.83:0/878673478 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f28480055e0 con 0x7f2854102a90 2024-04-25T11:43:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.119+0000 7f2851ffb700 1 --2- 172.21.15.83:0/878673478 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f28440a3cd0 0x7f28440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.119+0000 7f2851ffb700 1 -- 172.21.15.83:0/878673478 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f28480d04a0 con 0x7f2854102a90 2024-04-25T11:43:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.119+0000 7f2853fff700 1 --2- 172.21.15.83:0/878673478 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f28440a3cd0 0x7f28440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:04.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.119+0000 7f2853fff700 1 --2- 172.21.15.83:0/878673478 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f28440a3cd0 0x7f28440a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f284000b5c0 tx=0x7f2840005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:04.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.124+0000 7f2851ffb700 1 -- 172.21.15.83:0/878673478 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2848020020 con 0x7f2854102a90 2024-04-25T11:43:04.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.370+0000 7f285ac3f700 1 -- 172.21.15.83:0/878673478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f285402df80 con 0x7f2854102a90 2024-04-25T11:43:04.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.370+0000 7f2851ffb700 1 -- 172.21.15.83:0/878673478 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f284809cd30 con 0x7f2854102a90 2024-04-25T11:43:04.358 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:04.358 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T11:43:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.372+0000 7f285ac3f700 1 -- 172.21.15.83:0/878673478 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f28440a3cd0 msgr2=0x7f28440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.372+0000 7f285ac3f700 1 --2- 172.21.15.83:0/878673478 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f28440a3cd0 0x7f28440a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f284000b5c0 tx=0x7f2840005fb0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.372+0000 7f285ac3f700 1 -- 172.21.15.83:0/878673478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2854102a90 msgr2=0x7f2854199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.372+0000 7f285ac3f700 1 --2- 172.21.15.83:0/878673478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2854102a90 0x7f2854199200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f284800d980 tx=0x7f284800dd40 comp rx=0 tx=0).stop 2024-04-25T11:43:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.372+0000 7f285ac3f700 1 -- 172.21.15.83:0/878673478 shutdown_connections 2024-04-25T11:43:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.372+0000 7f285ac3f700 1 --2- 172.21.15.83:0/878673478 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f28440a3cd0 0x7f28440a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.372+0000 7f285ac3f700 1 --2- 172.21.15.83:0/878673478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2854108a50 0x7f285419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.372+0000 7f285ac3f700 1 --2- 172.21.15.83:0/878673478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2854102a90 0x7f2854199200 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.372+0000 7f285ac3f700 1 --2- 172.21.15.83:0/878673478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2854103450 0x7f2854199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.372+0000 7f285ac3f700 1 -- 172.21.15.83:0/878673478 >> 172.21.15.83:0/878673478 conn(0x7f2854079ba0 msgr2=0x7f28541018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.372+0000 7f285ac3f700 1 -- 172.21.15.83:0/878673478 shutdown_connections 2024-04-25T11:43:04.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:04.372+0000 7f285ac3f700 1 -- 172.21.15.83:0/878673478 wait complete. 2024-04-25T11:43:04.997 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:04 smithi083 ceph-mon[29330]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T11:43:04.998 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:04 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/878673478' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T11:43:05.025 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T11:43:05.026 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:43:05.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.139+0000 7fd46ce35640 1 -- 172.21.15.83:0/1101392495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd468100ea0 msgr2=0x7fd4681012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.139+0000 7fd46ce35640 1 --2- 172.21.15.83:0/1101392495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd468100ea0 0x7fd4681012c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd458009900 tx=0x7fd45802a410 comp rx=0 tx=0).stop 2024-04-25T11:43:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.139+0000 7fd46ce35640 1 -- 172.21.15.83:0/1101392495 shutdown_connections 2024-04-25T11:43:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.139+0000 7fd46ce35640 1 --2- 172.21.15.83:0/1101392495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4681029b0 0x7fd4681092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.139+0000 7fd46ce35640 1 --2- 172.21.15.83:0/1101392495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd468101ff0 0x7fd468102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.139+0000 7fd46ce35640 1 --2- 172.21.15.83:0/1101392495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd468100ea0 0x7fd4681012c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.139+0000 7fd46ce35640 1 -- 172.21.15.83:0/1101392495 >> 172.21.15.83:0/1101392495 conn(0x7fd4680fc430 msgr2=0x7fd4680fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.139+0000 7fd46ce35640 1 -- 172.21.15.83:0/1101392495 shutdown_connections 2024-04-25T11:43:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.139+0000 7fd46ce35640 1 -- 172.21.15.83:0/1101392495 wait complete. 2024-04-25T11:43:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.140+0000 7fd46ce35640 1 Processor -- start 2024-04-25T11:43:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.140+0000 7fd46ce35640 1 -- start start 2024-04-25T11:43:05.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.140+0000 7fd46ce35640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd468100ea0 0x7fd468198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.140+0000 7fd46ce35640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd468101ff0 0x7fd468198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.140+0000 7fd46ce35640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4681029b0 0x7fd46819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.140+0000 7fd46ce35640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd46810ca90 con 0x7fd4681029b0 2024-04-25T11:43:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.140+0000 7fd46ce35640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd46810c910 con 0x7fd468100ea0 2024-04-25T11:43:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.140+0000 7fd46ce35640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd46810cc10 con 0x7fd468101ff0 2024-04-25T11:43:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.140+0000 7fd466d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4681029b0 0x7fd46819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.140+0000 7fd466d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4681029b0 0x7fd46819f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60842/0 (socket says 172.21.15.83:60842) 2024-04-25T11:43:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.140+0000 7fd466d9d640 1 -- 172.21.15.83:0/3155065832 learned_addr learned my addr 172.21.15.83:0/3155065832 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.140+0000 7fd465d9b640 1 --2- 172.21.15.83:0/3155065832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd468101ff0 0x7fd468198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.141+0000 7fd46659c640 1 --2- 172.21.15.83:0/3155065832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd468100ea0 0x7fd468198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.141+0000 7fd466d9d640 1 -- 172.21.15.83:0/3155065832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd468101ff0 msgr2=0x7fd468198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.141+0000 7fd466d9d640 1 --2- 172.21.15.83:0/3155065832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd468101ff0 0x7fd468198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.141+0000 7fd466d9d640 1 -- 172.21.15.83:0/3155065832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd468100ea0 msgr2=0x7fd468198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.141+0000 7fd466d9d640 1 --2- 172.21.15.83:0/3155065832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd468100ea0 0x7fd468198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.141+0000 7fd466d9d640 1 -- 172.21.15.83:0/3155065832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46819fef0 con 0x7fd4681029b0 2024-04-25T11:43:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.141+0000 7fd46659c640 1 --2- 172.21.15.83:0/3155065832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd468100ea0 0x7fd468198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.141+0000 7fd466d9d640 1 --2- 172.21.15.83:0/3155065832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4681029b0 0x7fd46819f880 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd46000eaa0 tx=0x7fd46000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.141+0000 7fd4577fe640 1 -- 172.21.15.83:0/3155065832 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460004d20 con 0x7fd4681029b0 2024-04-25T11:43:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.141+0000 7fd4577fe640 1 -- 172.21.15.83:0/3155065832 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd460005020 con 0x7fd4681029b0 2024-04-25T11:43:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.141+0000 7fd46ce35640 1 -- 172.21.15.83:0/3155065832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4681a01e0 con 0x7fd4681029b0 2024-04-25T11:43:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.141+0000 7fd4577fe640 1 -- 172.21.15.83:0/3155065832 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4600116b0 con 0x7fd4681029b0 2024-04-25T11:43:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.141+0000 7fd46ce35640 1 -- 172.21.15.83:0/3155065832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4681a07b0 con 0x7fd4681029b0 2024-04-25T11:43:05.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.142+0000 7fd46ce35640 1 -- 172.21.15.83:0/3155065832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4681a0a70 con 0x7fd4681029b0 2024-04-25T11:43:05.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.143+0000 7fd4577fe640 1 -- 172.21.15.83:0/3155065832 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd4600051d0 con 0x7fd4681029b0 2024-04-25T11:43:05.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.144+0000 7fd4577fe640 1 --2- 172.21.15.83:0/3155065832 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd4500a3c40 0x7fd4500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.144+0000 7fd46659c640 1 --2- 172.21.15.83:0/3155065832 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd4500a3c40 0x7fd4500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.144+0000 7fd4577fe640 1 -- 172.21.15.83:0/3155065832 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd4600d07c0 con 0x7fd4681029b0 2024-04-25T11:43:05.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.144+0000 7fd46659c640 1 --2- 172.21.15.83:0/3155065832 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd4500a3c40 0x7fd4500a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd458006010 tx=0x7fd458000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:05.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.148+0000 7fd4577fe640 1 -- 172.21.15.83:0/3155065832 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd460016030 con 0x7fd4681029b0 2024-04-25T11:43:05.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:04 smithi094 ceph-mon[33032]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T11:43:05.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:04 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/878673478' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T11:43:05.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.327+0000 7fd46ce35640 1 -- 172.21.15.83:0/3155065832 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4681a0c40 con 0x7fd4681029b0 2024-04-25T11:43:05.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.328+0000 7fd4577fe640 1 -- 172.21.15.83:0/3155065832 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22581 (secure 0 0 0) 0x7fd46009d050 con 0x7fd4681029b0 2024-04-25T11:43:05.316 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:05.317 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:42:51.074495+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:41:42.546991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:41:43.764226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:43:05.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.330+0000 7fd46ce35640 1 -- 172.21.15.83:0/3155065832 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd4500a3c40 msgr2=0x7fd4500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.330+0000 7fd46ce35640 1 --2- 172.21.15.83:0/3155065832 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd4500a3c40 0x7fd4500a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd458006010 tx=0x7fd458000bc0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.330+0000 7fd46ce35640 1 -- 172.21.15.83:0/3155065832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4681029b0 msgr2=0x7fd46819f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.330+0000 7fd46ce35640 1 --2- 172.21.15.83:0/3155065832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4681029b0 0x7fd46819f880 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd46000eaa0 tx=0x7fd46000eea0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.330+0000 7fd46ce35640 1 -- 172.21.15.83:0/3155065832 shutdown_connections 2024-04-25T11:43:05.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.330+0000 7fd46ce35640 1 --2- 172.21.15.83:0/3155065832 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd4500a3c40 0x7fd4500a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.330+0000 7fd46ce35640 1 --2- 172.21.15.83:0/3155065832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4681029b0 0x7fd46819f880 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.330+0000 7fd46ce35640 1 --2- 172.21.15.83:0/3155065832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd468101ff0 0x7fd468198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.330+0000 7fd46ce35640 1 --2- 172.21.15.83:0/3155065832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd468100ea0 0x7fd468198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.330+0000 7fd46ce35640 1 -- 172.21.15.83:0/3155065832 >> 172.21.15.83:0/3155065832 conn(0x7fd4680fc430 msgr2=0x7fd46806a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:05.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.331+0000 7fd46ce35640 1 -- 172.21.15.83:0/3155065832 shutdown_connections 2024-04-25T11:43:05.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.331+0000 7fd46ce35640 1 -- 172.21.15.83:0/3155065832 wait complete. 2024-04-25T11:43:05.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:04 smithi111 ceph-mon[32784]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T11:43:05.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:04 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/878673478' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T11:43:05.329 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:37:15.914779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:41:42.546991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:41:43.764226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:43:05.330 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:43:05.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.443+0000 7fe4e462a640 1 -- 172.21.15.83:0/4209762385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc102a00 msgr2=0x7fe4dc109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.443+0000 7fe4e462a640 1 --2- 172.21.15.83:0/4209762385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc102a00 0x7fe4dc109320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009900 tx=0x7fe4cc02a410 comp rx=0 tx=0).stop 2024-04-25T11:43:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.443+0000 7fe4e462a640 1 -- 172.21.15.83:0/4209762385 shutdown_connections 2024-04-25T11:43:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.443+0000 7fe4e462a640 1 --2- 172.21.15.83:0/4209762385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc102a00 0x7fe4dc109320 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.443+0000 7fe4e462a640 1 --2- 172.21.15.83:0/4209762385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4dc102040 0x7fe4dc1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.443+0000 7fe4e462a640 1 --2- 172.21.15.83:0/4209762385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4dc100e60 0x7fe4dc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.443+0000 7fe4e462a640 1 -- 172.21.15.83:0/4209762385 >> 172.21.15.83:0/4209762385 conn(0x7fe4dc0fc420 msgr2=0x7fe4dc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.444+0000 7fe4e462a640 1 -- 172.21.15.83:0/4209762385 shutdown_connections 2024-04-25T11:43:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.444+0000 7fe4e462a640 1 -- 172.21.15.83:0/4209762385 wait complete. 2024-04-25T11:43:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.444+0000 7fe4e462a640 1 Processor -- start 2024-04-25T11:43:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.444+0000 7fe4e462a640 1 -- start start 2024-04-25T11:43:05.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.444+0000 7fe4e462a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4dc100e60 0x7fe4dc06b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.444+0000 7fe4e462a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc102040 0x7fe4dc19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.444+0000 7fe4e462a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4dc102a00 0x7fe4dc19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.444+0000 7fe4e462a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc10ca10 con 0x7fe4dc102040 2024-04-25T11:43:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.444+0000 7fe4e462a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc10c890 con 0x7fe4dc100e60 2024-04-25T11:43:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.444+0000 7fe4e462a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc10cb90 con 0x7fe4dc102a00 2024-04-25T11:43:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.444+0000 7fe4e1bc5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc102040 0x7fe4dc19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.444+0000 7fe4e1bc5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc102040 0x7fe4dc19ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60854/0 (socket says 172.21.15.83:60854) 2024-04-25T11:43:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.444+0000 7fe4e1bc5640 1 -- 172.21.15.83:0/462122697 learned_addr learned my addr 172.21.15.83:0/462122697 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.444+0000 7fe4e2bc7640 1 --2- 172.21.15.83:0/462122697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4dc102a00 0x7fe4dc19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.444+0000 7fe4e23c6640 1 --2- 172.21.15.83:0/462122697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4dc100e60 0x7fe4dc06b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.445+0000 7fe4e1bc5640 1 -- 172.21.15.83:0/462122697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4dc102a00 msgr2=0x7fe4dc19f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.445+0000 7fe4e1bc5640 1 --2- 172.21.15.83:0/462122697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4dc102a00 0x7fe4dc19f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.445+0000 7fe4e1bc5640 1 -- 172.21.15.83:0/462122697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4dc100e60 msgr2=0x7fe4dc06b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.445+0000 7fe4e1bc5640 1 --2- 172.21.15.83:0/462122697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4dc100e60 0x7fe4dc06b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.445+0000 7fe4e1bc5640 1 -- 172.21.15.83:0/462122697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc19fdb0 con 0x7fe4dc102040 2024-04-25T11:43:05.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.445+0000 7fe4e1bc5640 1 --2- 172.21.15.83:0/462122697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc102040 0x7fe4dc19ceb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000d950 tx=0x7fe4d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:05.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.445+0000 7fe4db7fe640 1 -- 172.21.15.83:0/462122697 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0017070 con 0x7fe4dc102040 2024-04-25T11:43:05.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.445+0000 7fe4db7fe640 1 -- 172.21.15.83:0/462122697 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d0009c30 con 0x7fe4dc102040 2024-04-25T11:43:05.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.445+0000 7fe4e462a640 1 -- 172.21.15.83:0/462122697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4dc1a00a0 con 0x7fe4dc102040 2024-04-25T11:43:05.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.445+0000 7fe4db7fe640 1 -- 172.21.15.83:0/462122697 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000f620 con 0x7fe4dc102040 2024-04-25T11:43:05.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.445+0000 7fe4e462a640 1 -- 172.21.15.83:0/462122697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4dc1a05f0 con 0x7fe4dc102040 2024-04-25T11:43:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.446+0000 7fe4e462a640 1 -- 172.21.15.83:0/462122697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4dc108280 con 0x7fe4dc102040 2024-04-25T11:43:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.447+0000 7fe4db7fe640 1 -- 172.21.15.83:0/462122697 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe4d0010460 con 0x7fe4dc102040 2024-04-25T11:43:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.447+0000 7fe4db7fe640 1 --2- 172.21.15.83:0/462122697 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe4c40a3c40 0x7fe4c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.447+0000 7fe4e23c6640 1 --2- 172.21.15.83:0/462122697 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe4c40a3c40 0x7fe4c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.448+0000 7fe4db7fe640 1 -- 172.21.15.83:0/462122697 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fe4d00d01b0 con 0x7fe4dc102040 2024-04-25T11:43:05.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.448+0000 7fe4e23c6640 1 --2- 172.21.15.83:0/462122697 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe4c40a3c40 0x7fe4c40a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc101bb0 tx=0x7fe4d4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:05.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.452+0000 7fe4db7fe640 1 -- 172.21.15.83:0/462122697 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4d009ca40 con 0x7fe4dc102040 2024-04-25T11:43:05.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.630+0000 7fe4e462a640 1 -- 172.21.15.83:0/462122697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe4dc02d5c0 con 0x7fe4dc102040 2024-04-25T11:43:05.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.630+0000 7fe4db7fe640 1 -- 172.21.15.83:0/462122697 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fe4d0009de0 con 0x7fe4dc102040 2024-04-25T11:43:05.618 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T11:43:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.632+0000 7fe4e462a640 1 -- 172.21.15.83:0/462122697 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe4c40a3c40 msgr2=0x7fe4c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.632+0000 7fe4e462a640 1 --2- 172.21.15.83:0/462122697 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe4c40a3c40 0x7fe4c40a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc101bb0 tx=0x7fe4d4009500 comp rx=0 tx=0).stop 2024-04-25T11:43:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.632+0000 7fe4e462a640 1 -- 172.21.15.83:0/462122697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc102040 msgr2=0x7fe4dc19ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.632+0000 7fe4e462a640 1 --2- 172.21.15.83:0/462122697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc102040 0x7fe4dc19ceb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000d950 tx=0x7fe4d000de10 comp rx=0 tx=0).stop 2024-04-25T11:43:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.632+0000 7fe4e462a640 1 -- 172.21.15.83:0/462122697 shutdown_connections 2024-04-25T11:43:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.632+0000 7fe4e462a640 1 --2- 172.21.15.83:0/462122697 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe4c40a3c40 0x7fe4c40a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.632+0000 7fe4e462a640 1 --2- 172.21.15.83:0/462122697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4dc102040 0x7fe4dc19ceb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.632+0000 7fe4e462a640 1 --2- 172.21.15.83:0/462122697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4dc102a00 0x7fe4dc19f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.632+0000 7fe4e462a640 1 --2- 172.21.15.83:0/462122697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4dc100e60 0x7fe4dc06b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.632+0000 7fe4e462a640 1 -- 172.21.15.83:0/462122697 >> 172.21.15.83:0/462122697 conn(0x7fe4dc0fc420 msgr2=0x7fe4dc105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.632+0000 7fe4e462a640 1 -- 172.21.15.83:0/462122697 shutdown_connections 2024-04-25T11:43:05.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.632+0000 7fe4e462a640 1 -- 172.21.15.83:0/462122697 wait complete. 2024-04-25T11:43:05.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:43:05.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.744+0000 7f42d8b8e640 1 -- 172.21.15.83:0/2991781749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d4101fc0 msgr2=0x7f42d4102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.744+0000 7f42d8b8e640 1 --2- 172.21.15.83:0/2991781749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d4101fc0 0x7f42d4102440 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009900 tx=0x7f42c402a3f0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.745+0000 7f42d8b8e640 1 -- 172.21.15.83:0/2991781749 shutdown_connections 2024-04-25T11:43:05.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.745+0000 7f42d8b8e640 1 --2- 172.21.15.83:0/2991781749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d4101fc0 0x7f42d4102440 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.745+0000 7f42d8b8e640 1 --2- 172.21.15.83:0/2991781749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42d4102980 0x7f42d41092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.745+0000 7f42d8b8e640 1 --2- 172.21.15.83:0/2991781749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42d4100e70 0x7f42d4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.745+0000 7f42d8b8e640 1 -- 172.21.15.83:0/2991781749 >> 172.21.15.83:0/2991781749 conn(0x7f42d40fc3e0 msgr2=0x7f42d40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.745+0000 7f42d8b8e640 1 -- 172.21.15.83:0/2991781749 shutdown_connections 2024-04-25T11:43:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.745+0000 7f42d8b8e640 1 -- 172.21.15.83:0/2991781749 wait complete. 2024-04-25T11:43:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.745+0000 7f42d8b8e640 1 Processor -- start 2024-04-25T11:43:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.745+0000 7f42d8b8e640 1 -- start start 2024-04-25T11:43:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.745+0000 7f42d8b8e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d4100e70 0x7f42d4069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.745+0000 7f42d8b8e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42d4101fc0 0x7f42d419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.745+0000 7f42d8b8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42d4102980 0x7f42d419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.745+0000 7f42d8b8e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42d410c9d0 con 0x7f42d4100e70 2024-04-25T11:43:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.745+0000 7f42d8b8e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f42d410c850 con 0x7f42d4102980 2024-04-25T11:43:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42d8b8e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f42d410cb50 con 0x7f42d4101fc0 2024-04-25T11:43:05.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42d259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d4100e70 0x7f42d4069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42d259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d4100e70 0x7f42d4069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60868/0 (socket says 172.21.15.83:60868) 2024-04-25T11:43:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42d259c640 1 -- 172.21.15.83:0/2889487470 learned_addr learned my addr 172.21.15.83:0/2889487470 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42d1d9b640 1 --2- 172.21.15.83:0/2889487470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42d4101fc0 0x7f42d419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42d259c640 1 -- 172.21.15.83:0/2889487470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42d4101fc0 msgr2=0x7f42d419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42d2d9d640 1 --2- 172.21.15.83:0/2889487470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42d4102980 0x7f42d419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42d259c640 1 --2- 172.21.15.83:0/2889487470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42d4101fc0 0x7f42d419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42d259c640 1 -- 172.21.15.83:0/2889487470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42d4102980 msgr2=0x7f42d419f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42d259c640 1 --2- 172.21.15.83:0/2889487470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42d4102980 0x7f42d419f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42d259c640 1 -- 172.21.15.83:0/2889487470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d419fd80 con 0x7f42d4100e70 2024-04-25T11:43:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42d2d9d640 1 --2- 172.21.15.83:0/2889487470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42d4102980 0x7f42d419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42d259c640 1 --2- 172.21.15.83:0/2889487470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d4100e70 0x7f42d4069c70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00c920 tx=0x7f42bc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42cb7fe640 1 -- 172.21.15.83:0/2889487470 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc007c70 con 0x7f42d4100e70 2024-04-25T11:43:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42cb7fe640 1 -- 172.21.15.83:0/2889487470 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42bc007e10 con 0x7f42d4100e70 2024-04-25T11:43:05.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42d8b8e640 1 -- 172.21.15.83:0/2889487470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d41a0070 con 0x7f42d4100e70 2024-04-25T11:43:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42cb7fe640 1 -- 172.21.15.83:0/2889487470 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc018820 con 0x7f42d4100e70 2024-04-25T11:43:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.746+0000 7f42d8b8e640 1 -- 172.21.15.83:0/2889487470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d41a05c0 con 0x7f42d4100e70 2024-04-25T11:43:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.747+0000 7f42d8b8e640 1 -- 172.21.15.83:0/2889487470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d4108250 con 0x7f42d4100e70 2024-04-25T11:43:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.748+0000 7f42cb7fe640 1 -- 172.21.15.83:0/2889487470 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f42bc00f450 con 0x7f42d4100e70 2024-04-25T11:43:05.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.749+0000 7f42cb7fe640 1 --2- 172.21.15.83:0/2889487470 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42c00a3c40 0x7f42c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.749+0000 7f42d1d9b640 1 --2- 172.21.15.83:0/2889487470 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42c00a3c40 0x7f42c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.749+0000 7f42cb7fe640 1 -- 172.21.15.83:0/2889487470 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f42bc0d0740 con 0x7f42d4100e70 2024-04-25T11:43:05.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.749+0000 7f42d1d9b640 1 --2- 172.21.15.83:0/2889487470 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42c00a3c40 0x7f42c00a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f42c4031500 tx=0x7f42c4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:05.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.753+0000 7f42cb7fe640 1 -- 172.21.15.83:0/2889487470 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f42bc014030 con 0x7f42d4100e70 2024-04-25T11:43:05.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.929+0000 7f42d8b8e640 1 -- 172.21.15.83:0/2889487470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f42d402d820 con 0x7f42d4100e70 2024-04-25T11:43:05.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.930+0000 7f42cb7fe640 1 -- 172.21.15.83:0/2889487470 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f42bc09cfd0 con 0x7f42d4100e70 2024-04-25T11:43:05.917 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T11:43:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.931+0000 7f42d8b8e640 1 -- 172.21.15.83:0/2889487470 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42c00a3c40 msgr2=0x7f42c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.931+0000 7f42d8b8e640 1 --2- 172.21.15.83:0/2889487470 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42c00a3c40 0x7f42c00a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f42c4031500 tx=0x7f42c4000bc0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.931+0000 7f42d8b8e640 1 -- 172.21.15.83:0/2889487470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d4100e70 msgr2=0x7f42d4069c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.931+0000 7f42d8b8e640 1 --2- 172.21.15.83:0/2889487470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d4100e70 0x7f42d4069c70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00c920 tx=0x7f42bc00cde0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.931+0000 7f42d8b8e640 1 -- 172.21.15.83:0/2889487470 shutdown_connections 2024-04-25T11:43:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.931+0000 7f42d8b8e640 1 --2- 172.21.15.83:0/2889487470 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f42c00a3c40 0x7f42c00a6100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.931+0000 7f42d8b8e640 1 --2- 172.21.15.83:0/2889487470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f42d4102980 0x7f42d419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.932+0000 7f42d8b8e640 1 --2- 172.21.15.83:0/2889487470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42d4101fc0 0x7f42d419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.932+0000 7f42d8b8e640 1 --2- 172.21.15.83:0/2889487470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d4100e70 0x7f42d4069c70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:05.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.932+0000 7f42d8b8e640 1 -- 172.21.15.83:0/2889487470 >> 172.21.15.83:0/2889487470 conn(0x7f42d40fc3e0 msgr2=0x7f42d4105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:05.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.932+0000 7f42d8b8e640 1 -- 172.21.15.83:0/2889487470 shutdown_connections 2024-04-25T11:43:05.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:05.932+0000 7f42d8b8e640 1 -- 172.21.15.83:0/2889487470 wait complete. 2024-04-25T11:43:05.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:43:06.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.044+0000 7fc5ee02a640 1 -- 172.21.15.83:0/3764033343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e8107c70 msgr2=0x7fc5e810a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.044+0000 7fc5ee02a640 1 --2- 172.21.15.83:0/3764033343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e8107c70 0x7fc5e810a0c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009ca0 tx=0x7fc5e002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.044+0000 7fc5ee02a640 1 -- 172.21.15.83:0/3764033343 shutdown_connections 2024-04-25T11:43:06.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.044+0000 7fc5ee02a640 1 --2- 172.21.15.83:0/3764033343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e8107c70 0x7fc5e810a0c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.044+0000 7fc5ee02a640 1 --2- 172.21.15.83:0/3764033343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e8105340 0x7fc5e8107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.044+0000 7fc5ee02a640 1 --2- 172.21.15.83:0/3764033343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5e8102a10 0x7fc5e8104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.044+0000 7fc5ee02a640 1 -- 172.21.15.83:0/3764033343 >> 172.21.15.83:0/3764033343 conn(0x7fc5e80fc440 msgr2=0x7fc5e80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:06.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.044+0000 7fc5ee02a640 1 -- 172.21.15.83:0/3764033343 shutdown_connections 2024-04-25T11:43:06.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.044+0000 7fc5ee02a640 1 -- 172.21.15.83:0/3764033343 wait complete. 2024-04-25T11:43:06.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5ee02a640 1 Processor -- start 2024-04-25T11:43:06.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5ee02a640 1 -- start start 2024-04-25T11:43:06.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5ee02a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e8102a10 0x7fc5e8198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5ee02a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e8105340 0x7fc5e81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5ee02a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5e8107c70 0x7fc5e819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5ee02a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e810db20 con 0x7fc5e8102a10 2024-04-25T11:43:06.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5ee02a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e810d9a0 con 0x7fc5e8107c70 2024-04-25T11:43:06.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5ee02a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e810dca0 con 0x7fc5e8105340 2024-04-25T11:43:06.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5e77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e8102a10 0x7fc5e8198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5e77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e8102a10 0x7fc5e8198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60888/0 (socket says 172.21.15.83:60888) 2024-04-25T11:43:06.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5e77fe640 1 -- 172.21.15.83:0/1719442620 learned_addr learned my addr 172.21.15.83:0/1719442620 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:06.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5e7fff640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5e8107c70 0x7fc5e819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5e6ffd640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e8105340 0x7fc5e81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5e77fe640 1 -- 172.21.15.83:0/1719442620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e8105340 msgr2=0x7fc5e81986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5e77fe640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e8105340 0x7fc5e81986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5e77fe640 1 -- 172.21.15.83:0/1719442620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5e8107c70 msgr2=0x7fc5e819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5e77fe640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5e8107c70 0x7fc5e819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5e77fe640 1 -- 172.21.15.83:0/1719442620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e819fd80 con 0x7fc5e8102a10 2024-04-25T11:43:06.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5e7fff640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5e8107c70 0x7fc5e819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:06.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5e6ffd640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e8105340 0x7fc5e81986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:06.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.045+0000 7fc5e77fe640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e8102a10 0x7fc5e8198160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800c920 tx=0x7fc5d800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:06.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.046+0000 7fc5e4ff9640 1 -- 172.21.15.83:0/1719442620 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8007c70 con 0x7fc5e8102a10 2024-04-25T11:43:06.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.046+0000 7fc5e4ff9640 1 -- 172.21.15.83:0/1719442620 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5d8007e10 con 0x7fc5e8102a10 2024-04-25T11:43:06.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.046+0000 7fc5ee02a640 1 -- 172.21.15.83:0/1719442620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e81a0070 con 0x7fc5e8102a10 2024-04-25T11:43:06.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.046+0000 7fc5e4ff9640 1 -- 172.21.15.83:0/1719442620 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8018820 con 0x7fc5e8102a10 2024-04-25T11:43:06.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.046+0000 7fc5ee02a640 1 -- 172.21.15.83:0/1719442620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e81a05c0 con 0x7fc5e8102a10 2024-04-25T11:43:06.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.047+0000 7fc5ee02a640 1 -- 172.21.15.83:0/1719442620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e80fdf30 con 0x7fc5e8102a10 2024-04-25T11:43:06.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.048+0000 7fc5e4ff9640 1 -- 172.21.15.83:0/1719442620 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc5d800f450 con 0x7fc5e8102a10 2024-04-25T11:43:06.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.048+0000 7fc5e4ff9640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc5d00a3c40 0x7fc5d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.048+0000 7fc5e6ffd640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc5d00a3c40 0x7fc5d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.048+0000 7fc5e4ff9640 1 -- 172.21.15.83:0/1719442620 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc5d80d0740 con 0x7fc5e8102a10 2024-04-25T11:43:06.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.049+0000 7fc5e6ffd640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc5d00a3c40 0x7fc5d00a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0058c0 tx=0x7fc5dc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:06.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.053+0000 7fc5e4ff9640 1 -- 172.21.15.83:0/1719442620 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5d8014030 con 0x7fc5e8102a10 2024-04-25T11:43:06.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.229+0000 7fc5ee02a640 1 -- 172.21.15.83:0/1719442620 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc5e8062b50 con 0x7fc5e8102a10 2024-04-25T11:43:06.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.230+0000 7fc5e4ff9640 1 -- 172.21.15.83:0/1719442620 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fc5d809cfd0 con 0x7fc5e8102a10 2024-04-25T11:43:06.217 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T11:43:06.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.231+0000 7fc5ee02a640 1 -- 172.21.15.83:0/1719442620 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc5d00a3c40 msgr2=0x7fc5d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.231+0000 7fc5ee02a640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc5d00a3c40 0x7fc5d00a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0058c0 tx=0x7fc5dc005850 comp rx=0 tx=0).stop 2024-04-25T11:43:06.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.231+0000 7fc5ee02a640 1 -- 172.21.15.83:0/1719442620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e8102a10 msgr2=0x7fc5e8198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.231+0000 7fc5ee02a640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e8102a10 0x7fc5e8198160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800c920 tx=0x7fc5d800cde0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.232+0000 7fc5ee02a640 1 -- 172.21.15.83:0/1719442620 shutdown_connections 2024-04-25T11:43:06.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.232+0000 7fc5ee02a640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc5d00a3c40 0x7fc5d00a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.232+0000 7fc5ee02a640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc5e8107c70 0x7fc5e819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.232+0000 7fc5ee02a640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5e8105340 0x7fc5e81986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.232+0000 7fc5ee02a640 1 --2- 172.21.15.83:0/1719442620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5e8102a10 0x7fc5e8198160 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.232+0000 7fc5ee02a640 1 -- 172.21.15.83:0/1719442620 >> 172.21.15.83:0/1719442620 conn(0x7fc5e80fc440 msgr2=0x7fc5e8103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:06.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.232+0000 7fc5ee02a640 1 -- 172.21.15.83:0/1719442620 shutdown_connections 2024-04-25T11:43:06.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.232+0000 7fc5ee02a640 1 -- 172.21.15.83:0/1719442620 wait complete. 2024-04-25T11:43:06.229 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:05 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3155065832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:06.229 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:05 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/462122697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:43:06.229 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:05 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2889487470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:43:06.231 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:43:06.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:05 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3155065832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:06.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:05 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/462122697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:43:06.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:05 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2889487470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:43:06.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:05 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3155065832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:06.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:05 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/462122697' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:43:06.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:05 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2889487470' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:43:06.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.340+0000 7f0e1e595640 1 -- 172.21.15.83:0/524421705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e18107d30 msgr2=0x7f0e1810a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.340+0000 7f0e1e595640 1 --2- 172.21.15.83:0/524421705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e18107d30 0x7f0e1810a120 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009ca0 tx=0x7f0e1002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.340+0000 7f0e1e595640 1 -- 172.21.15.83:0/524421705 shutdown_connections 2024-04-25T11:43:06.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.340+0000 7f0e1e595640 1 --2- 172.21.15.83:0/524421705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e18107d30 0x7f0e1810a120 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.340+0000 7f0e1e595640 1 --2- 172.21.15.83:0/524421705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e18105400 0x7f0e181077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.340+0000 7f0e1e595640 1 --2- 172.21.15.83:0/524421705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e18102ad0 0x7f0e18104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.340+0000 7f0e1e595640 1 -- 172.21.15.83:0/524421705 >> 172.21.15.83:0/524421705 conn(0x7f0e180fc440 msgr2=0x7f0e180fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.340+0000 7f0e1e595640 1 -- 172.21.15.83:0/524421705 shutdown_connections 2024-04-25T11:43:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.340+0000 7f0e1e595640 1 -- 172.21.15.83:0/524421705 wait complete. 2024-04-25T11:43:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.341+0000 7f0e1e595640 1 Processor -- start 2024-04-25T11:43:06.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.341+0000 7f0e1e595640 1 -- start start 2024-04-25T11:43:06.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.341+0000 7f0e1e595640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e18102ad0 0x7f0e1819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.341+0000 7f0e1e595640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e18105400 0x7f0e1819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.341+0000 7f0e1e595640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e18107d30 0x7f0e181a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.341+0000 7f0e1e595640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1810db10 con 0x7f0e18107d30 2024-04-25T11:43:06.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.341+0000 7f0e1e595640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1810d990 con 0x7f0e18102ad0 2024-04-25T11:43:06.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.341+0000 7f0e1e595640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1810dc90 con 0x7f0e18105400 2024-04-25T11:43:06.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.341+0000 7f0e1cb32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e18107d30 0x7f0e181a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.341+0000 7f0e1cb32640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e18107d30 0x7f0e181a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60896/0 (socket says 172.21.15.83:60896) 2024-04-25T11:43:06.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.341+0000 7f0e1cb32640 1 -- 172.21.15.83:0/3852841554 learned_addr learned my addr 172.21.15.83:0/3852841554 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:06.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.342+0000 7f0e177fe640 1 --2- 172.21.15.83:0/3852841554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e18105400 0x7f0e1819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.342+0000 7f0e17fff640 1 --2- 172.21.15.83:0/3852841554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e18102ad0 0x7f0e1819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.342+0000 7f0e1cb32640 1 -- 172.21.15.83:0/3852841554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e18105400 msgr2=0x7f0e1819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.342+0000 7f0e1cb32640 1 --2- 172.21.15.83:0/3852841554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e18105400 0x7f0e1819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.342+0000 7f0e1cb32640 1 -- 172.21.15.83:0/3852841554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e18102ad0 msgr2=0x7f0e1819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.342+0000 7f0e1cb32640 1 --2- 172.21.15.83:0/3852841554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e18102ad0 0x7f0e1819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.342+0000 7f0e1cb32640 1 -- 172.21.15.83:0/3852841554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e181a1f20 con 0x7f0e18107d30 2024-04-25T11:43:06.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.342+0000 7f0e177fe640 1 --2- 172.21.15.83:0/3852841554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e18105400 0x7f0e1819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:06.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.342+0000 7f0e1cb32640 1 --2- 172.21.15.83:0/3852841554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e18107d30 0x7f0e181a18b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0e100097b0 tx=0x7f0e10003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:06.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.342+0000 7f0e157fa640 1 -- 172.21.15.83:0/3852841554 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e10004240 con 0x7f0e18107d30 2024-04-25T11:43:06.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.342+0000 7f0e1e595640 1 -- 172.21.15.83:0/3852841554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e181a21b0 con 0x7f0e18107d30 2024-04-25T11:43:06.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.342+0000 7f0e157fa640 1 -- 172.21.15.83:0/3852841554 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e100317c0 con 0x7f0e18107d30 2024-04-25T11:43:06.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.342+0000 7f0e157fa640 1 -- 172.21.15.83:0/3852841554 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1002fd80 con 0x7f0e18107d30 2024-04-25T11:43:06.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.342+0000 7f0e1e595640 1 -- 172.21.15.83:0/3852841554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e181a2750 con 0x7f0e18107d30 2024-04-25T11:43:06.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.343+0000 7f0e1e595640 1 -- 172.21.15.83:0/3852841554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e18109820 con 0x7f0e18107d30 2024-04-25T11:43:06.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.344+0000 7f0e157fa640 1 -- 172.21.15.83:0/3852841554 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0e100043e0 con 0x7f0e18107d30 2024-04-25T11:43:06.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.345+0000 7f0e157fa640 1 --2- 172.21.15.83:0/3852841554 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0e000a3cc0 0x7f0e000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.345+0000 7f0e17fff640 1 --2- 172.21.15.83:0/3852841554 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0e000a3cc0 0x7f0e000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.345+0000 7f0e157fa640 1 -- 172.21.15.83:0/3852841554 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f0e100f1dd0 con 0x7f0e18107d30 2024-04-25T11:43:06.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.345+0000 7f0e17fff640 1 --2- 172.21.15.83:0/3852841554 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0e000a3cc0 0x7f0e000a6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800ba40 tx=0x7f0e08005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:06.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.349+0000 7f0e157fa640 1 -- 172.21.15.83:0/3852841554 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e100f5030 con 0x7f0e18107d30 2024-04-25T11:43:06.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.568+0000 7f0e1e595640 1 -- 172.21.15.83:0/3852841554 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e1802d790 con 0x7f0e18107d30 2024-04-25T11:43:06.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.569+0000 7f0e157fa640 1 -- 172.21.15.83:0/3852841554 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7f0e100be710 con 0x7f0e18107d30 2024-04-25T11:43:06.556 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:06.556 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:41:44.787926+0000","modified":"2024-04-25T11:41:51.578220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24469,"mds_2":24442,"mds_3":24440,"mds_4":24430},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/1568064835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1568064835},{"type":"v1","addr":"172.21.15.83:6839","nonce":1568064835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/1576544313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1576544313},{"type":"v1","addr":"172.21.15.83:6841","nonce":1576544313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":4,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.94:6837/2475976592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2475976592},{"type":"v1","addr":"172.21.15.94:6837","nonce":2475976592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.111:6837/2927430878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2927430878},{"type":"v1","addr":"172.21.15.111:6837","nonce":2927430878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/2233531151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2233531151},{"type":"v1","addr":"172.21.15.111:6839","nonce":2233531151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/1461078471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1461078471},{"type":"v1","addr":"172.21.15.94:6839","nonce":1461078471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24440,24430,24479,24469,24442]},"id":1}]} 2024-04-25T11:43:06.557 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-04-25T11:43:06.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.571+0000 7f0e1e595640 1 -- 172.21.15.83:0/3852841554 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0e000a3cc0 msgr2=0x7f0e000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.571+0000 7f0e1e595640 1 --2- 172.21.15.83:0/3852841554 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0e000a3cc0 0x7f0e000a6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800ba40 tx=0x7f0e08005cc0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.571+0000 7f0e1e595640 1 -- 172.21.15.83:0/3852841554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e18107d30 msgr2=0x7f0e181a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.571+0000 7f0e1e595640 1 --2- 172.21.15.83:0/3852841554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e18107d30 0x7f0e181a18b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0e100097b0 tx=0x7f0e10003c20 comp rx=0 tx=0).stop 2024-04-25T11:43:06.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.571+0000 7f0e1e595640 1 -- 172.21.15.83:0/3852841554 shutdown_connections 2024-04-25T11:43:06.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.571+0000 7f0e1e595640 1 --2- 172.21.15.83:0/3852841554 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0e000a3cc0 0x7f0e000a6180 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.571+0000 7f0e1e595640 1 --2- 172.21.15.83:0/3852841554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e18107d30 0x7f0e181a18b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.571+0000 7f0e1e595640 1 --2- 172.21.15.83:0/3852841554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e18105400 0x7f0e1819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.571+0000 7f0e1e595640 1 --2- 172.21.15.83:0/3852841554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e18102ad0 0x7f0e1819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.571+0000 7f0e1e595640 1 -- 172.21.15.83:0/3852841554 >> 172.21.15.83:0/3852841554 conn(0x7f0e180fc440 msgr2=0x7f0e18102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:06.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.571+0000 7f0e1e595640 1 -- 172.21.15.83:0/3852841554 shutdown_connections 2024-04-25T11:43:06.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.571+0000 7f0e1e595640 1 -- 172.21.15.83:0/3852841554 wait complete. 2024-04-25T11:43:06.570 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:43:06.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 -- 172.21.15.83:0/2473691444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27601029a0 msgr2=0x7f27601092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 --2- 172.21.15.83:0/2473691444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27601029a0 0x7f27601092c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2758009ca0 tx=0x7f275802a8c0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 -- 172.21.15.83:0/2473691444 shutdown_connections 2024-04-25T11:43:06.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 --2- 172.21.15.83:0/2473691444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27601029a0 0x7f27601092c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 --2- 172.21.15.83:0/2473691444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2760101fe0 0x7f2760102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 --2- 172.21.15.83:0/2473691444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2760100e90 0x7f27601012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 -- 172.21.15.83:0/2473691444 >> 172.21.15.83:0/2473691444 conn(0x7f27600fc400 msgr2=0x7f27600fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:06.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 -- 172.21.15.83:0/2473691444 shutdown_connections 2024-04-25T11:43:06.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 -- 172.21.15.83:0/2473691444 wait complete. 2024-04-25T11:43:06.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 Processor -- start 2024-04-25T11:43:06.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 -- start start 2024-04-25T11:43:06.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2760100e90 0x7f2760198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2760101fe0 0x7f27601986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27601029a0 0x7f276019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f276010db30 con 0x7f2760100e90 2024-04-25T11:43:06.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f276010d9b0 con 0x7f27601029a0 2024-04-25T11:43:06.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.681+0000 7f2765dad640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f276010dcb0 con 0x7f2760101fe0 2024-04-25T11:43:06.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f275f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2760100e90 0x7f2760198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f275f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2760100e90 0x7f2760198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60924/0 (socket says 172.21.15.83:60924) 2024-04-25T11:43:06.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f275f7fe640 1 -- 172.21.15.83:0/2944754948 learned_addr learned my addr 172.21.15.83:0/2944754948 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f2757fff640 1 --2- 172.21.15.83:0/2944754948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2760101fe0 0x7f27601986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f275ffff640 1 --2- 172.21.15.83:0/2944754948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27601029a0 0x7f276019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f275f7fe640 1 -- 172.21.15.83:0/2944754948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2760101fe0 msgr2=0x7f27601986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f275f7fe640 1 --2- 172.21.15.83:0/2944754948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2760101fe0 0x7f27601986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f275f7fe640 1 -- 172.21.15.83:0/2944754948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27601029a0 msgr2=0x7f276019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f275f7fe640 1 --2- 172.21.15.83:0/2944754948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27601029a0 0x7f276019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f275f7fe640 1 -- 172.21.15.83:0/2944754948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276019fdb0 con 0x7f2760100e90 2024-04-25T11:43:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f275ffff640 1 --2- 172.21.15.83:0/2944754948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27601029a0 0x7f276019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f275f7fe640 1 --2- 172.21.15.83:0/2944754948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2760100e90 0x7f2760198190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f275000d770 tx=0x7f275000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f275d7fa640 1 -- 172.21.15.83:0/2944754948 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750017070 con 0x7f2760100e90 2024-04-25T11:43:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f2765dad640 1 -- 172.21.15.83:0/2944754948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27601a00a0 con 0x7f2760100e90 2024-04-25T11:43:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f275d7fa640 1 -- 172.21.15.83:0/2944754948 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27500054d0 con 0x7f2760100e90 2024-04-25T11:43:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f275d7fa640 1 -- 172.21.15.83:0/2944754948 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750004c30 con 0x7f2760100e90 2024-04-25T11:43:06.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.682+0000 7f2765dad640 1 -- 172.21.15.83:0/2944754948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27601a05a0 con 0x7f2760100e90 2024-04-25T11:43:06.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.683+0000 7f2765dad640 1 -- 172.21.15.83:0/2944754948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27601081a0 con 0x7f2760100e90 2024-04-25T11:43:06.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.684+0000 7f275d7fa640 1 -- 172.21.15.83:0/2944754948 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2750004e10 con 0x7f2760100e90 2024-04-25T11:43:06.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.685+0000 7f275d7fa640 1 --2- 172.21.15.83:0/2944754948 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f274c0a3c40 0x7f274c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.685+0000 7f2757fff640 1 --2- 172.21.15.83:0/2944754948 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f274c0a3c40 0x7f274c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.685+0000 7f275d7fa640 1 -- 172.21.15.83:0/2944754948 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f27500cffa0 con 0x7f2760100e90 2024-04-25T11:43:06.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.685+0000 7f2757fff640 1 --2- 172.21.15.83:0/2944754948 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f274c0a3c40 0x7f274c0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f27601996a0 tx=0x7f274800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:06.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.689+0000 7f275d7fa640 1 -- 172.21.15.83:0/2944754948 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f275009c8d0 con 0x7f2760100e90 2024-04-25T11:43:06.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.870+0000 7f2765dad640 1 -- 172.21.15.83:0/2944754948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2760062b50 con 0x7f2760100e90 2024-04-25T11:43:06.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.871+0000 7f275d7fa640 1 -- 172.21.15.83:0/2944754948 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22581 (secure 0 0 0) 0x7f275009c1d0 con 0x7f2760100e90 2024-04-25T11:43:06.859 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:06.859 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:42:51.074495+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:41:42.546991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:41:43.764226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:43:06.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.873+0000 7f2765dad640 1 -- 172.21.15.83:0/2944754948 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f274c0a3c40 msgr2=0x7f274c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.873+0000 7f2765dad640 1 --2- 172.21.15.83:0/2944754948 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f274c0a3c40 0x7f274c0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f27601996a0 tx=0x7f274800b390 comp rx=0 tx=0).stop 2024-04-25T11:43:06.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.873+0000 7f2765dad640 1 -- 172.21.15.83:0/2944754948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2760100e90 msgr2=0x7f2760198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.873+0000 7f2765dad640 1 --2- 172.21.15.83:0/2944754948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2760100e90 0x7f2760198190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f275000d770 tx=0x7f275000dc30 comp rx=0 tx=0).stop 2024-04-25T11:43:06.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.873+0000 7f2765dad640 1 -- 172.21.15.83:0/2944754948 shutdown_connections 2024-04-25T11:43:06.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.873+0000 7f2765dad640 1 --2- 172.21.15.83:0/2944754948 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f274c0a3c40 0x7f274c0a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.873+0000 7f2765dad640 1 --2- 172.21.15.83:0/2944754948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27601029a0 0x7f276019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.873+0000 7f2765dad640 1 --2- 172.21.15.83:0/2944754948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2760101fe0 0x7f27601986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.873+0000 7f2765dad640 1 --2- 172.21.15.83:0/2944754948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2760100e90 0x7f2760198190 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.873+0000 7f2765dad640 1 -- 172.21.15.83:0/2944754948 >> 172.21.15.83:0/2944754948 conn(0x7f27600fc400 msgr2=0x7f2760105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:06.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.873+0000 7f2765dad640 1 -- 172.21.15.83:0/2944754948 shutdown_connections 2024-04-25T11:43:06.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.873+0000 7f2765dad640 1 -- 172.21.15.83:0/2944754948 wait complete. 2024-04-25T11:43:06.872 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:37:15.914779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:41:42.546991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:41:43.764226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:43:06.873 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:43:06.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.986+0000 7fc25bf4f640 1 -- 172.21.15.83:0/2243807650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc254107c70 msgr2=0x7fc25410a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.986+0000 7fc25bf4f640 1 --2- 172.21.15.83:0/2243807650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc254107c70 0x7fc25410a0c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009ca0 tx=0x7fc24c02a8c0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.986+0000 7fc25bf4f640 1 -- 172.21.15.83:0/2243807650 shutdown_connections 2024-04-25T11:43:06.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.986+0000 7fc25bf4f640 1 --2- 172.21.15.83:0/2243807650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc254107c70 0x7fc25410a0c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.986+0000 7fc25bf4f640 1 --2- 172.21.15.83:0/2243807650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc254105340 0x7fc254107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.986+0000 7fc25bf4f640 1 --2- 172.21.15.83:0/2243807650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc254102a10 0x7fc254104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.986+0000 7fc25bf4f640 1 -- 172.21.15.83:0/2243807650 >> 172.21.15.83:0/2243807650 conn(0x7fc2540fc400 msgr2=0x7fc2540fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:06.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.986+0000 7fc25bf4f640 1 -- 172.21.15.83:0/2243807650 shutdown_connections 2024-04-25T11:43:06.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.986+0000 7fc25bf4f640 1 -- 172.21.15.83:0/2243807650 wait complete. 2024-04-25T11:43:06.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.986+0000 7fc25bf4f640 1 Processor -- start 2024-04-25T11:43:06.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25bf4f640 1 -- start start 2024-04-25T11:43:06.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25bf4f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc254102a10 0x7fc254198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25bf4f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc254105340 0x7fc2541986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25bf4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc254107c70 0x7fc25419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25bf4f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc25410dae0 con 0x7fc254107c70 2024-04-25T11:43:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25bf4f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc25410d960 con 0x7fc254105340 2024-04-25T11:43:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25bf4f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc25410dc60 con 0x7fc254102a10 2024-04-25T11:43:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25a4ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc254107c70 0x7fc25419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25a4ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc254107c70 0x7fc25419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60938/0 (socket says 172.21.15.83:60938) 2024-04-25T11:43:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25a4ec640 1 -- 172.21.15.83:0/1596559254 learned_addr learned my addr 172.21.15.83:0/1596559254 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc2594ea640 1 --2- 172.21.15.83:0/1596559254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc254105340 0x7fc2541986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc259ceb640 1 --2- 172.21.15.83:0/1596559254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc254102a10 0x7fc254198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25a4ec640 1 -- 172.21.15.83:0/1596559254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc254102a10 msgr2=0x7fc254198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25a4ec640 1 --2- 172.21.15.83:0/1596559254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc254102a10 0x7fc254198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25a4ec640 1 -- 172.21.15.83:0/1596559254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc254105340 msgr2=0x7fc2541986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25a4ec640 1 --2- 172.21.15.83:0/1596559254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc254105340 0x7fc2541986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25a4ec640 1 -- 172.21.15.83:0/1596559254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25419fda0 con 0x7fc254107c70 2024-04-25T11:43:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25a4ec640 1 --2- 172.21.15.83:0/1596559254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc254107c70 0x7fc25419f730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0097b0 tx=0x7fc24c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:06.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc24affd640 1 -- 172.21.15.83:0/1596559254 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c004240 con 0x7fc254107c70 2024-04-25T11:43:06.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc24affd640 1 -- 172.21.15.83:0/1596559254 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc24c0317c0 con 0x7fc254107c70 2024-04-25T11:43:06.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25bf4f640 1 -- 172.21.15.83:0/1596559254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2541a0030 con 0x7fc254107c70 2024-04-25T11:43:06.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc24affd640 1 -- 172.21.15.83:0/1596559254 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c02fd80 con 0x7fc254107c70 2024-04-25T11:43:06.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.987+0000 7fc25bf4f640 1 -- 172.21.15.83:0/1596559254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2541a0580 con 0x7fc254107c70 2024-04-25T11:43:06.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.988+0000 7fc25bf4f640 1 -- 172.21.15.83:0/1596559254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2540fdef0 con 0x7fc254107c70 2024-04-25T11:43:06.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.990+0000 7fc24affd640 1 -- 172.21.15.83:0/1596559254 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc24c0043e0 con 0x7fc254107c70 2024-04-25T11:43:06.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.990+0000 7fc24affd640 1 --2- 172.21.15.83:0/1596559254 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc2400a3c40 0x7fc2400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:06.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.990+0000 7fc259ceb640 1 --2- 172.21.15.83:0/1596559254 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc2400a3c40 0x7fc2400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:06.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.990+0000 7fc24affd640 1 -- 172.21.15.83:0/1596559254 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fc24c0f1fd0 con 0x7fc254107c70 2024-04-25T11:43:06.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.991+0000 7fc259ceb640 1 --2- 172.21.15.83:0/1596559254 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc2400a3c40 0x7fc2400a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc250005fd0 tx=0x7fc250005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:06.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:06.995+0000 7fc24affd640 1 -- 172.21.15.83:0/1596559254 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc24c0bea90 con 0x7fc254107c70 2024-04-25T11:43:07.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:06 smithi083 ceph-mon[29330]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T11:43:07.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:06 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1719442620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:43:07.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:06 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3852841554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:43:07.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:06 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2944754948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:07.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.172+0000 7fc25bf4f640 1 -- 172.21.15.83:0/1596559254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc25402d5c0 con 0x7fc254107c70 2024-04-25T11:43:07.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.173+0000 7fc24affd640 1 -- 172.21.15.83:0/1596559254 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fc24c0be390 con 0x7fc254107c70 2024-04-25T11:43:07.160 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T11:43:07.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.174+0000 7fc25bf4f640 1 -- 172.21.15.83:0/1596559254 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc2400a3c40 msgr2=0x7fc2400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.174+0000 7fc25bf4f640 1 --2- 172.21.15.83:0/1596559254 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc2400a3c40 0x7fc2400a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc250005fd0 tx=0x7fc250005850 comp rx=0 tx=0).stop 2024-04-25T11:43:07.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.174+0000 7fc25bf4f640 1 -- 172.21.15.83:0/1596559254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc254107c70 msgr2=0x7fc25419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.174+0000 7fc25bf4f640 1 --2- 172.21.15.83:0/1596559254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc254107c70 0x7fc25419f730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0097b0 tx=0x7fc24c003c20 comp rx=0 tx=0).stop 2024-04-25T11:43:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.175+0000 7fc25bf4f640 1 -- 172.21.15.83:0/1596559254 shutdown_connections 2024-04-25T11:43:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.175+0000 7fc25bf4f640 1 --2- 172.21.15.83:0/1596559254 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fc2400a3c40 0x7fc2400a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.175+0000 7fc25bf4f640 1 --2- 172.21.15.83:0/1596559254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc254107c70 0x7fc25419f730 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.175+0000 7fc25bf4f640 1 --2- 172.21.15.83:0/1596559254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc254105340 0x7fc2541986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.175+0000 7fc25bf4f640 1 --2- 172.21.15.83:0/1596559254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc254102a10 0x7fc254198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.175+0000 7fc25bf4f640 1 -- 172.21.15.83:0/1596559254 >> 172.21.15.83:0/1596559254 conn(0x7fc2540fc400 msgr2=0x7fc254103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.175+0000 7fc25bf4f640 1 -- 172.21.15.83:0/1596559254 shutdown_connections 2024-04-25T11:43:07.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.175+0000 7fc25bf4f640 1 -- 172.21.15.83:0/1596559254 wait complete. 2024-04-25T11:43:07.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:43:07.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:06 smithi094 ceph-mon[33032]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T11:43:07.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:06 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1719442620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:43:07.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:06 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3852841554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:43:07.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:06 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2944754948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:07.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.286+0000 7f61f787b640 1 -- 172.21.15.83:0/3515175793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f0101fc0 msgr2=0x7f61f0102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.286+0000 7f61f787b640 1 --2- 172.21.15.83:0/3515175793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f0101fc0 0x7f61f0102440 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f61ec009900 tx=0x7f61ec02a410 comp rx=0 tx=0).stop 2024-04-25T11:43:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.286+0000 7f61f787b640 1 -- 172.21.15.83:0/3515175793 shutdown_connections 2024-04-25T11:43:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.286+0000 7f61f787b640 1 --2- 172.21.15.83:0/3515175793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f0101fc0 0x7f61f0102440 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.286+0000 7f61f787b640 1 --2- 172.21.15.83:0/3515175793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f0102980 0x7f61f01092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.286+0000 7f61f787b640 1 --2- 172.21.15.83:0/3515175793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61f0100e70 0x7f61f0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.286+0000 7f61f787b640 1 -- 172.21.15.83:0/3515175793 >> 172.21.15.83:0/3515175793 conn(0x7f61f00fc3e0 msgr2=0x7f61f00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.287+0000 7f61f787b640 1 -- 172.21.15.83:0/3515175793 shutdown_connections 2024-04-25T11:43:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.287+0000 7f61f787b640 1 -- 172.21.15.83:0/3515175793 wait complete. 2024-04-25T11:43:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.287+0000 7f61f787b640 1 Processor -- start 2024-04-25T11:43:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.287+0000 7f61f787b640 1 -- start start 2024-04-25T11:43:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.287+0000 7f61f787b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61f0100e70 0x7f61f0069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.287+0000 7f61f787b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f0101fc0 0x7f61f019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.287+0000 7f61f787b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f0102980 0x7f61f019f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.287+0000 7f61f787b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61f010c9c0 con 0x7f61f0102980 2024-04-25T11:43:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.287+0000 7f61f787b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61f010c840 con 0x7f61f0101fc0 2024-04-25T11:43:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.287+0000 7f61f787b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f61f010cb40 con 0x7f61f0100e70 2024-04-25T11:43:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.287+0000 7f61f5e18640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f0102980 0x7f61f019f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.287+0000 7f61f5e18640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f0102980 0x7f61f019f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60964/0 (socket says 172.21.15.83:60964) 2024-04-25T11:43:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.287+0000 7f61f5e18640 1 -- 172.21.15.83:0/757437177 learned_addr learned my addr 172.21.15.83:0/757437177 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.288+0000 7f61f4e16640 1 --2- 172.21.15.83:0/757437177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f0101fc0 0x7f61f019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.288+0000 7f61f5617640 1 --2- 172.21.15.83:0/757437177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61f0100e70 0x7f61f0069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.288+0000 7f61f5e18640 1 -- 172.21.15.83:0/757437177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61f0100e70 msgr2=0x7f61f0069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.288+0000 7f61f5e18640 1 --2- 172.21.15.83:0/757437177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61f0100e70 0x7f61f0069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.288+0000 7f61f5e18640 1 -- 172.21.15.83:0/757437177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f0101fc0 msgr2=0x7f61f019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.288+0000 7f61f5e18640 1 --2- 172.21.15.83:0/757437177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f0101fc0 0x7f61f019cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.288+0000 7f61f5e18640 1 -- 172.21.15.83:0/757437177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f019fdf0 con 0x7f61f0102980 2024-04-25T11:43:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.288+0000 7f61f5e18640 1 --2- 172.21.15.83:0/757437177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f0102980 0x7f61f019f530 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f61e800eb10 tx=0x7f61e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:07.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.288+0000 7f61e67fc640 1 -- 172.21.15.83:0/757437177 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8004b20 con 0x7f61f0102980 2024-04-25T11:43:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.288+0000 7f61e67fc640 1 -- 172.21.15.83:0/757437177 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61e8004cc0 con 0x7f61f0102980 2024-04-25T11:43:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.288+0000 7f61f787b640 1 -- 172.21.15.83:0/757437177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f01a00e0 con 0x7f61f0102980 2024-04-25T11:43:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.288+0000 7f61e67fc640 1 -- 172.21.15.83:0/757437177 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8010670 con 0x7f61f0102980 2024-04-25T11:43:07.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.288+0000 7f61f787b640 1 -- 172.21.15.83:0/757437177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f01a05e0 con 0x7f61f0102980 2024-04-25T11:43:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.289+0000 7f61f787b640 1 -- 172.21.15.83:0/757437177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f0108250 con 0x7f61f0102980 2024-04-25T11:43:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.290+0000 7f61e67fc640 1 -- 172.21.15.83:0/757437177 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f61e8005020 con 0x7f61f0102980 2024-04-25T11:43:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.291+0000 7f61e67fc640 1 --2- 172.21.15.83:0/757437177 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f61dc0a3c40 0x7f61dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.291+0000 7f61f5617640 1 --2- 172.21.15.83:0/757437177 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f61dc0a3c40 0x7f61dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.291+0000 7f61e67fc640 1 -- 172.21.15.83:0/757437177 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f61e80d0c60 con 0x7f61f0102980 2024-04-25T11:43:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.291+0000 7f61f5617640 1 --2- 172.21.15.83:0/757437177 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f61dc0a3c40 0x7f61dc0a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f61e0005fd0 tx=0x7f61e0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:07.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.295+0000 7f61e67fc640 1 -- 172.21.15.83:0/757437177 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f61e809d4f0 con 0x7f61f0102980 2024-04-25T11:43:07.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:06 smithi111 ceph-mon[32784]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T11:43:07.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:06 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1719442620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:43:07.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:06 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3852841554' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:43:07.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:06 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2944754948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:07.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.471+0000 7f61f787b640 1 -- 172.21.15.83:0/757437177 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f61f002d770 con 0x7f61f0102980 2024-04-25T11:43:07.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.472+0000 7f61e67fc640 1 -- 172.21.15.83:0/757437177 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f61e809cdf0 con 0x7f61f0102980 2024-04-25T11:43:07.459 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T11:43:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.474+0000 7f61f787b640 1 -- 172.21.15.83:0/757437177 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f61dc0a3c40 msgr2=0x7f61dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.474+0000 7f61f787b640 1 --2- 172.21.15.83:0/757437177 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f61dc0a3c40 0x7f61dc0a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f61e0005fd0 tx=0x7f61e0005850 comp rx=0 tx=0).stop 2024-04-25T11:43:07.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.474+0000 7f61f787b640 1 -- 172.21.15.83:0/757437177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f0102980 msgr2=0x7f61f019f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.474+0000 7f61f787b640 1 --2- 172.21.15.83:0/757437177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f0102980 0x7f61f019f530 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f61e800eb10 tx=0x7f61e800c2d0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.474+0000 7f61f787b640 1 -- 172.21.15.83:0/757437177 shutdown_connections 2024-04-25T11:43:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.474+0000 7f61f787b640 1 --2- 172.21.15.83:0/757437177 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f61dc0a3c40 0x7f61dc0a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.474+0000 7f61f787b640 1 --2- 172.21.15.83:0/757437177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61f0102980 0x7f61f019f530 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.474+0000 7f61f787b640 1 --2- 172.21.15.83:0/757437177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61f0101fc0 0x7f61f019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.474+0000 7f61f787b640 1 --2- 172.21.15.83:0/757437177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61f0100e70 0x7f61f0069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.474+0000 7f61f787b640 1 -- 172.21.15.83:0/757437177 >> 172.21.15.83:0/757437177 conn(0x7f61f00fc3e0 msgr2=0x7f61f0105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.474+0000 7f61f787b640 1 -- 172.21.15.83:0/757437177 shutdown_connections 2024-04-25T11:43:07.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.474+0000 7f61f787b640 1 -- 172.21.15.83:0/757437177 wait complete. 2024-04-25T11:43:07.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:43:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 -- 172.21.15.83:0/2531342251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728105340 msgr2=0x7f9728107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 --2- 172.21.15.83:0/2531342251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728105340 0x7f9728107730 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9718009900 tx=0x7f971802a410 comp rx=0 tx=0).stop 2024-04-25T11:43:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 -- 172.21.15.83:0/2531342251 shutdown_connections 2024-04-25T11:43:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 --2- 172.21.15.83:0/2531342251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728105340 0x7f9728107730 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 --2- 172.21.15.83:0/2531342251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9728107c70 0x7f972810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 --2- 172.21.15.83:0/2531342251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9728102a10 0x7f9728104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 -- 172.21.15.83:0/2531342251 >> 172.21.15.83:0/2531342251 conn(0x7f97280fc440 msgr2=0x7f97280fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 -- 172.21.15.83:0/2531342251 shutdown_connections 2024-04-25T11:43:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 -- 172.21.15.83:0/2531342251 wait complete. 2024-04-25T11:43:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 Processor -- start 2024-04-25T11:43:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 -- start start 2024-04-25T11:43:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9728102a10 0x7f9728198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728105340 0x7f9728198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9728107c70 0x7f972819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f972810daa0 con 0x7f9728105340 2024-04-25T11:43:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f972810d920 con 0x7f9728102a10 2024-04-25T11:43:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f97306fa640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f972810dc20 con 0x7f9728107c70 2024-04-25T11:43:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f972dc95640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728105340 0x7f9728198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f972dc95640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728105340 0x7f9728198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60978/0 (socket says 172.21.15.83:60978) 2024-04-25T11:43:07.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.586+0000 7f972dc95640 1 -- 172.21.15.83:0/1902541041 learned_addr learned my addr 172.21.15.83:0/1902541041 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.587+0000 7f972e496640 1 --2- 172.21.15.83:0/1902541041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9728102a10 0x7f9728198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.587+0000 7f972ec97640 1 --2- 172.21.15.83:0/1902541041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9728107c70 0x7f972819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.587+0000 7f972dc95640 1 -- 172.21.15.83:0/1902541041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9728107c70 msgr2=0x7f972819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.587+0000 7f972dc95640 1 --2- 172.21.15.83:0/1902541041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9728107c70 0x7f972819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.587+0000 7f972dc95640 1 -- 172.21.15.83:0/1902541041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9728102a10 msgr2=0x7f9728198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.587+0000 7f972dc95640 1 --2- 172.21.15.83:0/1902541041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9728102a10 0x7f9728198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.587+0000 7f972dc95640 1 -- 172.21.15.83:0/1902541041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972819fe00 con 0x7f9728105340 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.587+0000 7f972e496640 1 --2- 172.21.15.83:0/1902541041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9728102a10 0x7f9728198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.587+0000 7f972dc95640 1 --2- 172.21.15.83:0/1902541041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728105340 0x7f9728198770 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9718006010 tx=0x7f97180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.587+0000 7f971f7fe640 1 -- 172.21.15.83:0/1902541041 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971803f030 con 0x7f9728105340 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.587+0000 7f971f7fe640 1 -- 172.21.15.83:0/1902541041 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97180318f0 con 0x7f9728105340 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.587+0000 7f97306fa640 1 -- 172.21.15.83:0/1902541041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97281a0090 con 0x7f9728105340 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.587+0000 7f971f7fe640 1 -- 172.21.15.83:0/1902541041 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718035070 con 0x7f9728105340 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.587+0000 7f97306fa640 1 -- 172.21.15.83:0/1902541041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97281a0580 con 0x7f9728105340 2024-04-25T11:43:07.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.588+0000 7f97306fa640 1 -- 172.21.15.83:0/1902541041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97280fdf30 con 0x7f9728105340 2024-04-25T11:43:07.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.589+0000 7f971f7fe640 1 -- 172.21.15.83:0/1902541041 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9718042030 con 0x7f9728105340 2024-04-25T11:43:07.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.590+0000 7f971f7fe640 1 --2- 172.21.15.83:0/1902541041 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f97140a3c40 0x7f97140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.590+0000 7f972e496640 1 --2- 172.21.15.83:0/1902541041 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f97140a3c40 0x7f97140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.590+0000 7f971f7fe640 1 -- 172.21.15.83:0/1902541041 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f97180f0f80 con 0x7f9728105340 2024-04-25T11:43:07.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.590+0000 7f972e496640 1 --2- 172.21.15.83:0/1902541041 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f97140a3c40 0x7f97140a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9724009c20 tx=0x7f9724005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:07.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.594+0000 7f971f7fe640 1 -- 172.21.15.83:0/1902541041 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f97180bd810 con 0x7f9728105340 2024-04-25T11:43:07.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.770+0000 7f97306fa640 1 -- 172.21.15.83:0/1902541041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f972802d810 con 0x7f9728105340 2024-04-25T11:43:07.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.771+0000 7f971f7fe640 1 -- 172.21.15.83:0/1902541041 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f97180bd110 con 0x7f9728105340 2024-04-25T11:43:07.758 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T11:43:07.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.773+0000 7f97306fa640 1 -- 172.21.15.83:0/1902541041 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f97140a3c40 msgr2=0x7f97140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.773+0000 7f97306fa640 1 --2- 172.21.15.83:0/1902541041 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f97140a3c40 0x7f97140a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9724009c20 tx=0x7f9724005f90 comp rx=0 tx=0).stop 2024-04-25T11:43:07.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.773+0000 7f97306fa640 1 -- 172.21.15.83:0/1902541041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728105340 msgr2=0x7f9728198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.773+0000 7f97306fa640 1 --2- 172.21.15.83:0/1902541041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728105340 0x7f9728198770 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9718006010 tx=0x7f97180042f0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.773+0000 7f97306fa640 1 -- 172.21.15.83:0/1902541041 shutdown_connections 2024-04-25T11:43:07.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.773+0000 7f97306fa640 1 --2- 172.21.15.83:0/1902541041 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f97140a3c40 0x7f97140a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.773+0000 7f97306fa640 1 --2- 172.21.15.83:0/1902541041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9728105340 0x7f9728198770 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.773+0000 7f97306fa640 1 --2- 172.21.15.83:0/1902541041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9728107c70 0x7f972819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.773+0000 7f97306fa640 1 --2- 172.21.15.83:0/1902541041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9728102a10 0x7f9728198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.773+0000 7f97306fa640 1 -- 172.21.15.83:0/1902541041 >> 172.21.15.83:0/1902541041 conn(0x7f97280fc440 msgr2=0x7f9728103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:07.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.773+0000 7f97306fa640 1 -- 172.21.15.83:0/1902541041 shutdown_connections 2024-04-25T11:43:07.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.773+0000 7f97306fa640 1 -- 172.21.15.83:0/1902541041 wait complete. 2024-04-25T11:43:07.771 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:43:07.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.882+0000 7fbfbc7e8640 1 -- 172.21.15.83:0/1496360650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb4103190 msgr2=0x7fbfb4103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.882+0000 7fbfbc7e8640 1 --2- 172.21.15.83:0/1496360650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb4103190 0x7fbfb4103610 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009900 tx=0x7fbfa402a410 comp rx=0 tx=0).stop 2024-04-25T11:43:07.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.882+0000 7fbfbc7e8640 1 -- 172.21.15.83:0/1496360650 shutdown_connections 2024-04-25T11:43:07.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.882+0000 7fbfbc7e8640 1 --2- 172.21.15.83:0/1496360650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb4103190 0x7fbfb4103610 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.882+0000 7fbfbc7e8640 1 --2- 172.21.15.83:0/1496360650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfb4103b50 0x7fbfb410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.882+0000 7fbfbc7e8640 1 --2- 172.21.15.83:0/1496360650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb4101fa0 0x7fbfb41023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.882+0000 7fbfbc7e8640 1 -- 172.21.15.83:0/1496360650 >> 172.21.15.83:0/1496360650 conn(0x7fbfb40fd530 msgr2=0x7fbfb40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:07.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.882+0000 7fbfbc7e8640 1 -- 172.21.15.83:0/1496360650 shutdown_connections 2024-04-25T11:43:07.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.882+0000 7fbfbc7e8640 1 -- 172.21.15.83:0/1496360650 wait complete. 2024-04-25T11:43:07.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfbc7e8640 1 Processor -- start 2024-04-25T11:43:07.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfbc7e8640 1 -- start start 2024-04-25T11:43:07.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfbc7e8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb4101fa0 0x7fbfb4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfbc7e8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfb4103190 0x7fbfb4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfbc7e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb4103b50 0x7fbfb419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfbc7e8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb410db60 con 0x7fbfb4101fa0 2024-04-25T11:43:07.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfbc7e8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb410d9e0 con 0x7fbfb4103b50 2024-04-25T11:43:07.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfbc7e8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb410dce0 con 0x7fbfb4103190 2024-04-25T11:43:07.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfba584640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb4101fa0 0x7fbfb4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfba584640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb4101fa0 0x7fbfb4198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32778/0 (socket says 172.21.15.83:32778) 2024-04-25T11:43:07.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfba584640 1 -- 172.21.15.83:0/1613964539 learned_addr learned my addr 172.21.15.83:0/1613964539 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:07.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfb9d83640 1 --2- 172.21.15.83:0/1613964539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfb4103190 0x7fbfb4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfbad85640 1 --2- 172.21.15.83:0/1613964539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb4103b50 0x7fbfb419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfba584640 1 -- 172.21.15.83:0/1613964539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfb4103190 msgr2=0x7fbfb4198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfba584640 1 --2- 172.21.15.83:0/1613964539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfb4103190 0x7fbfb4198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfba584640 1 -- 172.21.15.83:0/1613964539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb4103b50 msgr2=0x7fbfb419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:07.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfba584640 1 --2- 172.21.15.83:0/1613964539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb4103b50 0x7fbfb419f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:07.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfba584640 1 -- 172.21.15.83:0/1613964539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb419fe50 con 0x7fbfb4101fa0 2024-04-25T11:43:07.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.883+0000 7fbfba584640 1 --2- 172.21.15.83:0/1613964539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb4101fa0 0x7fbfb4198230 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000b9c0 tx=0x7fbfb000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:07.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.884+0000 7fbfab7fe640 1 -- 172.21.15.83:0/1613964539 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0019070 con 0x7fbfb4101fa0 2024-04-25T11:43:07.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.884+0000 7fbfbc7e8640 1 -- 172.21.15.83:0/1613964539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb41a0140 con 0x7fbfb4101fa0 2024-04-25T11:43:07.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.884+0000 7fbfab7fe640 1 -- 172.21.15.83:0/1613964539 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb000cba0 con 0x7fbfb4101fa0 2024-04-25T11:43:07.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.884+0000 7fbfab7fe640 1 -- 172.21.15.83:0/1613964539 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb00125a0 con 0x7fbfb4101fa0 2024-04-25T11:43:07.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.884+0000 7fbfbc7e8640 1 -- 172.21.15.83:0/1613964539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb41a0760 con 0x7fbfb4101fa0 2024-04-25T11:43:07.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.885+0000 7fbfbc7e8640 1 -- 172.21.15.83:0/1613964539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb4109310 con 0x7fbfb4101fa0 2024-04-25T11:43:07.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.886+0000 7fbfab7fe640 1 -- 172.21.15.83:0/1613964539 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbfb001c070 con 0x7fbfb4101fa0 2024-04-25T11:43:07.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.886+0000 7fbfab7fe640 1 --2- 172.21.15.83:0/1613964539 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbfa00a3c40 0x7fbfa00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:07.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.886+0000 7fbfb9d83640 1 --2- 172.21.15.83:0/1613964539 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbfa00a3c40 0x7fbfa00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:07.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.886+0000 7fbfab7fe640 1 -- 172.21.15.83:0/1613964539 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fbfb00cfa80 con 0x7fbfb4101fa0 2024-04-25T11:43:07.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.887+0000 7fbfb9d83640 1 --2- 172.21.15.83:0/1613964539 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbfa00a3c40 0x7fbfa00a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4199740 tx=0x7fbfa40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:07.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:07.891+0000 7fbfab7fe640 1 -- 172.21.15.83:0/1613964539 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbfb009c420 con 0x7fbfb4101fa0 2024-04-25T11:43:08.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.109+0000 7fbfbc7e8640 1 -- 172.21.15.83:0/1613964539 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfb402d5c0 con 0x7fbfb4101fa0 2024-04-25T11:43:08.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.110+0000 7fbfab7fe640 1 -- 172.21.15.83:0/1613964539 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7fbfb009bd20 con 0x7fbfb4101fa0 2024-04-25T11:43:08.096 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:08.096 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:41:44.787926+0000","modified":"2024-04-25T11:41:51.578220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24469,"mds_2":24442,"mds_3":24440,"mds_4":24430},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/1568064835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1568064835},{"type":"v1","addr":"172.21.15.83:6839","nonce":1568064835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/1576544313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1576544313},{"type":"v1","addr":"172.21.15.83:6841","nonce":1576544313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":4,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.94:6837/2475976592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2475976592},{"type":"v1","addr":"172.21.15.94:6837","nonce":2475976592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.111:6837/2927430878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2927430878},{"type":"v1","addr":"172.21.15.111:6837","nonce":2927430878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/2233531151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2233531151},{"type":"v1","addr":"172.21.15.111:6839","nonce":2233531151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/1461078471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1461078471},{"type":"v1","addr":"172.21.15.94:6839","nonce":1461078471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24440,24430,24479,24469,24442]},"id":1}]} 2024-04-25T11:43:08.096 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-04-25T11:43:08.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.111+0000 7fbfbc7e8640 1 -- 172.21.15.83:0/1613964539 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbfa00a3c40 msgr2=0x7fbfa00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.111+0000 7fbfbc7e8640 1 --2- 172.21.15.83:0/1613964539 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbfa00a3c40 0x7fbfa00a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4199740 tx=0x7fbfa40314a0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.111+0000 7fbfbc7e8640 1 -- 172.21.15.83:0/1613964539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb4101fa0 msgr2=0x7fbfb4198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.111+0000 7fbfbc7e8640 1 --2- 172.21.15.83:0/1613964539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb4101fa0 0x7fbfb4198230 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000b9c0 tx=0x7fbfb000be80 comp rx=0 tx=0).stop 2024-04-25T11:43:08.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.112+0000 7fbfbc7e8640 1 -- 172.21.15.83:0/1613964539 shutdown_connections 2024-04-25T11:43:08.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.112+0000 7fbfbc7e8640 1 --2- 172.21.15.83:0/1613964539 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbfa00a3c40 0x7fbfa00a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.112+0000 7fbfbc7e8640 1 --2- 172.21.15.83:0/1613964539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbfb4103b50 0x7fbfb419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.112+0000 7fbfbc7e8640 1 --2- 172.21.15.83:0/1613964539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbfb4103190 0x7fbfb4198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.112+0000 7fbfbc7e8640 1 --2- 172.21.15.83:0/1613964539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbfb4101fa0 0x7fbfb4198230 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.112+0000 7fbfbc7e8640 1 -- 172.21.15.83:0/1613964539 >> 172.21.15.83:0/1613964539 conn(0x7fbfb40fd530 msgr2=0x7fbfb41064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:08.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.112+0000 7fbfbc7e8640 1 -- 172.21.15.83:0/1613964539 shutdown_connections 2024-04-25T11:43:08.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.112+0000 7fbfbc7e8640 1 -- 172.21.15.83:0/1613964539 wait complete. 2024-04-25T11:43:08.108 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:08 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1596559254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:43:08.108 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:08 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/757437177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:43:08.108 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:08 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1902541041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:43:08.109 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:43:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.220+0000 7f513c9a2640 1 -- 172.21.15.83:0/2688490618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5138102a00 msgr2=0x7f5138109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.220+0000 7f513c9a2640 1 --2- 172.21.15.83:0/2688490618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5138102a00 0x7f5138109320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5130009ca0 tx=0x7f513002a8c0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.220+0000 7f513c9a2640 1 -- 172.21.15.83:0/2688490618 shutdown_connections 2024-04-25T11:43:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.220+0000 7f513c9a2640 1 --2- 172.21.15.83:0/2688490618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5138102a00 0x7f5138109320 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.220+0000 7f513c9a2640 1 --2- 172.21.15.83:0/2688490618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5138102040 0x7f51381024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.220+0000 7f513c9a2640 1 --2- 172.21.15.83:0/2688490618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138100e60 0x7f5138101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.220+0000 7f513c9a2640 1 -- 172.21.15.83:0/2688490618 >> 172.21.15.83:0/2688490618 conn(0x7f51380fc440 msgr2=0x7f51380fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.220+0000 7f513c9a2640 1 -- 172.21.15.83:0/2688490618 shutdown_connections 2024-04-25T11:43:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.220+0000 7f513c9a2640 1 -- 172.21.15.83:0/2688490618 wait complete. 2024-04-25T11:43:08.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.221+0000 7f513c9a2640 1 Processor -- start 2024-04-25T11:43:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.221+0000 7f513c9a2640 1 -- start start 2024-04-25T11:43:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.221+0000 7f513c9a2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138100e60 0x7f5138198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.221+0000 7f513c9a2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5138102040 0x7f5138198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.221+0000 7f513c9a2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5138102a00 0x7f513819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.221+0000 7f513c9a2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f513810ca10 con 0x7f5138102a00 2024-04-25T11:43:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.221+0000 7f513c9a2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f513810c890 con 0x7f5138102040 2024-04-25T11:43:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.221+0000 7f513c9a2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f513810cb90 con 0x7f5138100e60 2024-04-25T11:43:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.221+0000 7f5136d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5138102a00 0x7f513819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.221+0000 7f5136d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5138102a00 0x7f513819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32804/0 (socket says 172.21.15.83:32804) 2024-04-25T11:43:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.221+0000 7f5136d9d640 1 -- 172.21.15.83:0/2787721898 learned_addr learned my addr 172.21.15.83:0/2787721898 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.221+0000 7f5135d9b640 1 --2- 172.21.15.83:0/2787721898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5138102040 0x7f5138198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.221+0000 7f513659c640 1 --2- 172.21.15.83:0/2787721898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138100e60 0x7f5138198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.222+0000 7f5136d9d640 1 -- 172.21.15.83:0/2787721898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138100e60 msgr2=0x7f5138198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.222+0000 7f5136d9d640 1 --2- 172.21.15.83:0/2787721898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138100e60 0x7f5138198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.222+0000 7f5136d9d640 1 -- 172.21.15.83:0/2787721898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5138102040 msgr2=0x7f5138198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.222+0000 7f5136d9d640 1 --2- 172.21.15.83:0/2787721898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5138102040 0x7f5138198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.222+0000 7f5136d9d640 1 -- 172.21.15.83:0/2787721898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513819fe60 con 0x7f5138102a00 2024-04-25T11:43:08.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.222+0000 7f513659c640 1 --2- 172.21.15.83:0/2787721898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138100e60 0x7f5138198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:08.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.222+0000 7f5136d9d640 1 --2- 172.21.15.83:0/2787721898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5138102a00 0x7f513819f7f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f51300097b0 tx=0x7f5130031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:08.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.222+0000 7f51277fe640 1 -- 172.21.15.83:0/2787721898 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5130003bf0 con 0x7f5138102a00 2024-04-25T11:43:08.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.222+0000 7f51277fe640 1 -- 172.21.15.83:0/2787721898 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5130003d90 con 0x7f5138102a00 2024-04-25T11:43:08.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.222+0000 7f513c9a2640 1 -- 172.21.15.83:0/2787721898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51381a00f0 con 0x7f5138102a00 2024-04-25T11:43:08.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.222+0000 7f51277fe640 1 -- 172.21.15.83:0/2787721898 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513002fc30 con 0x7f5138102a00 2024-04-25T11:43:08.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.222+0000 7f513c9a2640 1 -- 172.21.15.83:0/2787721898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51381a0640 con 0x7f5138102a00 2024-04-25T11:43:08.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.223+0000 7f513c9a2640 1 -- 172.21.15.83:0/2787721898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51381a08a0 con 0x7f5138102a00 2024-04-25T11:43:08.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.224+0000 7f51277fe640 1 -- 172.21.15.83:0/2787721898 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5130004040 con 0x7f5138102a00 2024-04-25T11:43:08.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.224+0000 7f51277fe640 1 --2- 172.21.15.83:0/2787721898 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f51200a3c40 0x7f51200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.225+0000 7f513659c640 1 --2- 172.21.15.83:0/2787721898 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f51200a3c40 0x7f51200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.225+0000 7f51277fe640 1 -- 172.21.15.83:0/2787721898 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f51300f1b40 con 0x7f5138102a00 2024-04-25T11:43:08.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.225+0000 7f513659c640 1 --2- 172.21.15.83:0/2787721898 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f51200a3c40 0x7f51200a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5138101bb0 tx=0x7f5128005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:08.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.229+0000 7f51277fe640 1 -- 172.21.15.83:0/2787721898 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51300be4e0 con 0x7f5138102a00 2024-04-25T11:43:08.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:08 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1596559254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:43:08.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:08 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/757437177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:43:08.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:08 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1902541041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:43:08.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:08 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1596559254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:43:08.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:08 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/757437177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:43:08.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:08 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1902541041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:43:08.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.407+0000 7f513c9a2640 1 -- 172.21.15.83:0/2787721898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51381a0f60 con 0x7f5138102a00 2024-04-25T11:43:08.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.408+0000 7f51277fe640 1 -- 172.21.15.83:0/2787721898 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22581 (secure 0 0 0) 0x7f51300bdde0 con 0x7f5138102a00 2024-04-25T11:43:08.395 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:08.395 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:42:51.074495+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:41:42.546991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:41:43.764226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:43:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.411+0000 7f513c9a2640 1 -- 172.21.15.83:0/2787721898 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f51200a3c40 msgr2=0x7f51200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.411+0000 7f513c9a2640 1 --2- 172.21.15.83:0/2787721898 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f51200a3c40 0x7f51200a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5138101bb0 tx=0x7f5128005f90 comp rx=0 tx=0).stop 2024-04-25T11:43:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.411+0000 7f513c9a2640 1 -- 172.21.15.83:0/2787721898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5138102a00 msgr2=0x7f513819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.411+0000 7f513c9a2640 1 --2- 172.21.15.83:0/2787721898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5138102a00 0x7f513819f7f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f51300097b0 tx=0x7f5130031840 comp rx=0 tx=0).stop 2024-04-25T11:43:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.411+0000 7f513c9a2640 1 -- 172.21.15.83:0/2787721898 shutdown_connections 2024-04-25T11:43:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.411+0000 7f513c9a2640 1 --2- 172.21.15.83:0/2787721898 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f51200a3c40 0x7f51200a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.411+0000 7f513c9a2640 1 --2- 172.21.15.83:0/2787721898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5138102a00 0x7f513819f7f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.411+0000 7f513c9a2640 1 --2- 172.21.15.83:0/2787721898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5138102040 0x7f5138198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.411+0000 7f513c9a2640 1 --2- 172.21.15.83:0/2787721898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5138100e60 0x7f5138198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.411+0000 7f513c9a2640 1 -- 172.21.15.83:0/2787721898 >> 172.21.15.83:0/2787721898 conn(0x7f51380fc440 msgr2=0x7f51380fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.411+0000 7f513c9a2640 1 -- 172.21.15.83:0/2787721898 shutdown_connections 2024-04-25T11:43:08.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.411+0000 7f513c9a2640 1 -- 172.21.15.83:0/2787721898 wait complete. 2024-04-25T11:43:08.409 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:43:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.521+0000 7f7998b08640 1 -- 172.21.15.83:0/3687826652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79941029b0 msgr2=0x7f79941092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.521+0000 7f7998b08640 1 --2- 172.21.15.83:0/3687826652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79941029b0 0x7f79941092d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7984009900 tx=0x7f798402a410 comp rx=0 tx=0).stop 2024-04-25T11:43:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.521+0000 7f7998b08640 1 -- 172.21.15.83:0/3687826652 shutdown_connections 2024-04-25T11:43:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.521+0000 7f7998b08640 1 --2- 172.21.15.83:0/3687826652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79941029b0 0x7f79941092d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.521+0000 7f7998b08640 1 --2- 172.21.15.83:0/3687826652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7994101ff0 0x7f7994102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.521+0000 7f7998b08640 1 --2- 172.21.15.83:0/3687826652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7994100ea0 0x7f79941012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.521+0000 7f7998b08640 1 -- 172.21.15.83:0/3687826652 >> 172.21.15.83:0/3687826652 conn(0x7f79940fc430 msgr2=0x7f79940fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.521+0000 7f7998b08640 1 -- 172.21.15.83:0/3687826652 shutdown_connections 2024-04-25T11:43:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.521+0000 7f7998b08640 1 -- 172.21.15.83:0/3687826652 wait complete. 2024-04-25T11:43:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7998b08640 1 Processor -- start 2024-04-25T11:43:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7998b08640 1 -- start start 2024-04-25T11:43:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7998b08640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7994100ea0 0x7f79941970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7998b08640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7994101ff0 0x7f79941975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7998b08640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79941029b0 0x7f799419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7998b08640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f799410c9f0 con 0x7f79941029b0 2024-04-25T11:43:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7998b08640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f799410c870 con 0x7f7994101ff0 2024-04-25T11:43:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7998b08640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f799410cb70 con 0x7f7994100ea0 2024-04-25T11:43:08.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7992d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79941029b0 0x7f799419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7992d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79941029b0 0x7f799419e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32820/0 (socket says 172.21.15.83:32820) 2024-04-25T11:43:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7992d9d640 1 -- 172.21.15.83:0/536464712 learned_addr learned my addr 172.21.15.83:0/536464712 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f799259c640 1 --2- 172.21.15.83:0/536464712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7994100ea0 0x7f79941970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7992d9d640 1 -- 172.21.15.83:0/536464712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7994100ea0 msgr2=0x7f79941970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7992d9d640 1 --2- 172.21.15.83:0/536464712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7994100ea0 0x7f79941970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7992d9d640 1 -- 172.21.15.83:0/536464712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7994101ff0 msgr2=0x7f79941975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7992d9d640 1 --2- 172.21.15.83:0/536464712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7994101ff0 0x7f79941975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7992d9d640 1 -- 172.21.15.83:0/536464712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799419ecc0 con 0x7f79941029b0 2024-04-25T11:43:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7992d9d640 1 --2- 172.21.15.83:0/536464712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79941029b0 0x7f799419e650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7984031500 tx=0x7f7984031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f79837fe640 1 -- 172.21.15.83:0/536464712 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798403f030 con 0x7f79941029b0 2024-04-25T11:43:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f79837fe640 1 -- 172.21.15.83:0/536464712 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7984003aa0 con 0x7f79941029b0 2024-04-25T11:43:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7998b08640 1 -- 172.21.15.83:0/536464712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799419ef50 con 0x7f79941029b0 2024-04-25T11:43:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f79837fe640 1 -- 172.21.15.83:0/536464712 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984035070 con 0x7f79941029b0 2024-04-25T11:43:08.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.522+0000 7f7998b08640 1 -- 172.21.15.83:0/536464712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799419f410 con 0x7f79941029b0 2024-04-25T11:43:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.524+0000 7f7998b08640 1 -- 172.21.15.83:0/536464712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79941081b0 con 0x7f79941029b0 2024-04-25T11:43:08.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.524+0000 7f79837fe640 1 -- 172.21.15.83:0/536464712 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7984042030 con 0x7f79941029b0 2024-04-25T11:43:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.525+0000 7f79837fe640 1 --2- 172.21.15.83:0/536464712 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f797c0a3c40 0x7f797c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.525+0000 7f799259c640 1 --2- 172.21.15.83:0/536464712 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f797c0a3c40 0x7f797c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.525+0000 7f79837fe640 1 -- 172.21.15.83:0/536464712 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f79840f0bf0 con 0x7f79941029b0 2024-04-25T11:43:08.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.525+0000 7f799259c640 1 --2- 172.21.15.83:0/536464712 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f797c0a3c40 0x7f797c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f798c0058c0 tx=0x7f798c005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:08.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.529+0000 7f79837fe640 1 -- 172.21.15.83:0/536464712 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79840bd480 con 0x7f79941029b0 2024-04-25T11:43:08.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.746+0000 7f7998b08640 1 -- 172.21.15.83:0/536464712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f799402d770 con 0x7f79941029b0 2024-04-25T11:43:08.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.747+0000 7f79837fe640 1 -- 172.21.15.83:0/536464712 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7f79840bcd80 con 0x7f79941029b0 2024-04-25T11:43:08.733 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:08.733 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:41:44.787926+0000","modified":"2024-04-25T11:41:51.578220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24469,"mds_2":24442,"mds_3":24440,"mds_4":24430},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/1568064835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1568064835},{"type":"v1","addr":"172.21.15.83:6839","nonce":1568064835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/1576544313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1576544313},{"type":"v1","addr":"172.21.15.83:6841","nonce":1576544313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":4,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.94:6837/2475976592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2475976592},{"type":"v1","addr":"172.21.15.94:6837","nonce":2475976592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.111:6837/2927430878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2927430878},{"type":"v1","addr":"172.21.15.111:6837","nonce":2927430878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/2233531151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2233531151},{"type":"v1","addr":"172.21.15.111:6839","nonce":2233531151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/1461078471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1461078471},{"type":"v1","addr":"172.21.15.94:6839","nonce":1461078471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24440,24430,24479,24469,24442]},"id":1}]} 2024-04-25T11:43:08.734 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-04-25T11:43:08.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.749+0000 7f7998b08640 1 -- 172.21.15.83:0/536464712 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f797c0a3c40 msgr2=0x7f797c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.749+0000 7f7998b08640 1 --2- 172.21.15.83:0/536464712 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f797c0a3c40 0x7f797c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f798c0058c0 tx=0x7f798c005c60 comp rx=0 tx=0).stop 2024-04-25T11:43:08.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.749+0000 7f7998b08640 1 -- 172.21.15.83:0/536464712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79941029b0 msgr2=0x7f799419e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.749+0000 7f7998b08640 1 --2- 172.21.15.83:0/536464712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79941029b0 0x7f799419e650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7984031500 tx=0x7f7984031a50 comp rx=0 tx=0).stop 2024-04-25T11:43:08.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.749+0000 7f7998b08640 1 -- 172.21.15.83:0/536464712 shutdown_connections 2024-04-25T11:43:08.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.749+0000 7f7998b08640 1 --2- 172.21.15.83:0/536464712 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f797c0a3c40 0x7f797c0a6100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.749+0000 7f7998b08640 1 --2- 172.21.15.83:0/536464712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79941029b0 0x7f799419e650 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.749+0000 7f7998b08640 1 --2- 172.21.15.83:0/536464712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7994101ff0 0x7f79941975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.749+0000 7f7998b08640 1 --2- 172.21.15.83:0/536464712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7994100ea0 0x7f79941970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.749+0000 7f7998b08640 1 -- 172.21.15.83:0/536464712 >> 172.21.15.83:0/536464712 conn(0x7f79940fc430 msgr2=0x7f7994105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:08.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.749+0000 7f7998b08640 1 -- 172.21.15.83:0/536464712 shutdown_connections 2024-04-25T11:43:08.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.749+0000 7f7998b08640 1 -- 172.21.15.83:0/536464712 wait complete. 2024-04-25T11:43:08.747 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T11:41:44.787926+0000', 'modified': '2024-04-25T11:41:51.578220+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24479, 'mds_1': 24469, 'mds_2': 24442, 'mds_3': 24440, 'mds_4': 24430}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14631': {'gid': 14631, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6839/1568064835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 1568064835}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 1568064835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6841/1576544313', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6840', 'nonce': 1576544313}, {'type': 'v1', 'addr': '172.21.15.83:6841', 'nonce': 1576544313}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24391': {'gid': 24391, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6833/2234401967', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 2234401967}, {'type': 'v1', 'addr': '172.21.15.111:6833', 'nonce': 2234401967}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24392': {'gid': 24392, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6833/3250237836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6832', 'nonce': 3250237836}, {'type': 'v1', 'addr': '172.21.15.94:6833', 'nonce': 3250237836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6835/4086957377', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6834', 'nonce': 4086957377}, {'type': 'v1', 'addr': '172.21.15.94:6835', 'nonce': 4086957377}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.111:6835/4167388778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 4167388778}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 4167388778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.94:6837/2475976592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6836', 'nonce': 2475976592}, {'type': 'v1', 'addr': '172.21.15.94:6837', 'nonce': 2475976592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.111:6837/2927430878', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 2927430878}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 2927430878}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.111:6839/2233531151', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 2233531151}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 2233531151}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.94:6839/1461078471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6838', 'nonce': 1461078471}, {'type': 'v1', 'addr': '172.21.15.94:6839', 'nonce': 1461078471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24479, 'qdb_cluster': [24440, 24430, 24479, 24469, 24442]} 2024-04-25T11:43:08.747 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T11:43:08.747 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T11:43:08.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.859+0000 7f1e53fff640 1 -- 172.21.15.83:0/3515362240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e54102850 msgr2=0x7f1e541090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.859+0000 7f1e53fff640 1 --2- 172.21.15.83:0/3515362240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e54102850 0x7f1e541090e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009900 tx=0x7f1e4402a410 comp rx=0 tx=0).stop 2024-04-25T11:43:08.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.859+0000 7f1e53fff640 1 -- 172.21.15.83:0/3515362240 shutdown_connections 2024-04-25T11:43:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.859+0000 7f1e53fff640 1 --2- 172.21.15.83:0/3515362240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e54102850 0x7f1e541090e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.859+0000 7f1e53fff640 1 --2- 172.21.15.83:0/3515362240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e54101e90 0x7f1e54102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.859+0000 7f1e53fff640 1 --2- 172.21.15.83:0/3515362240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e54100ca0 0x7f1e541010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.859+0000 7f1e53fff640 1 -- 172.21.15.83:0/3515362240 >> 172.21.15.83:0/3515362240 conn(0x7f1e540fc230 msgr2=0x7f1e540fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.860+0000 7f1e53fff640 1 -- 172.21.15.83:0/3515362240 shutdown_connections 2024-04-25T11:43:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.860+0000 7f1e53fff640 1 -- 172.21.15.83:0/3515362240 wait complete. 2024-04-25T11:43:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.860+0000 7f1e53fff640 1 Processor -- start 2024-04-25T11:43:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.860+0000 7f1e53fff640 1 -- start start 2024-04-25T11:43:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.860+0000 7f1e53fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e54100ca0 0x7f1e5406b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.860+0000 7f1e53fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e54101e90 0x7f1e5419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e53fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e54102850 0x7f1e5419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e53fff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5410c7d0 con 0x7f1e54101e90 2024-04-25T11:43:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e53fff640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5410c650 con 0x7f1e54100ca0 2024-04-25T11:43:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e53fff640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5410c950 con 0x7f1e54102850 2024-04-25T11:43:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e527fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e54101e90 0x7f1e5419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e527fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e54101e90 0x7f1e5419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32842/0 (socket says 172.21.15.83:32842) 2024-04-25T11:43:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e527fc640 1 -- 172.21.15.83:0/3646892107 learned_addr learned my addr 172.21.15.83:0/3646892107 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e52ffd640 1 --2- 172.21.15.83:0/3646892107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e54100ca0 0x7f1e5406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e537fe640 1 --2- 172.21.15.83:0/3646892107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e54102850 0x7f1e5419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e527fc640 1 -- 172.21.15.83:0/3646892107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e54102850 msgr2=0x7f1e5419f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e527fc640 1 --2- 172.21.15.83:0/3646892107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e54102850 0x7f1e5419f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e527fc640 1 -- 172.21.15.83:0/3646892107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e54100ca0 msgr2=0x7f1e5406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e527fc640 1 --2- 172.21.15.83:0/3646892107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e54100ca0 0x7f1e5406b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e527fc640 1 -- 172.21.15.83:0/3646892107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5419fba0 con 0x7f1e54101e90 2024-04-25T11:43:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e527fc640 1 --2- 172.21.15.83:0/3646892107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e54101e90 0x7f1e5419cc70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800d950 tx=0x7f1e4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e3bfff640 1 -- 172.21.15.83:0/3646892107 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48017070 con 0x7f1e54101e90 2024-04-25T11:43:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e53fff640 1 -- 172.21.15.83:0/3646892107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5419fe30 con 0x7f1e54101e90 2024-04-25T11:43:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e3bfff640 1 -- 172.21.15.83:0/3646892107 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e48009c30 con 0x7f1e54101e90 2024-04-25T11:43:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e3bfff640 1 -- 172.21.15.83:0/3646892107 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4800f620 con 0x7f1e54101e90 2024-04-25T11:43:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.861+0000 7f1e53fff640 1 -- 172.21.15.83:0/3646892107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e541a02f0 con 0x7f1e54101e90 2024-04-25T11:43:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.863+0000 7f1e3bfff640 1 -- 172.21.15.83:0/3646892107 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1e48010460 con 0x7f1e54101e90 2024-04-25T11:43:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e5c1ea740 1 Processor -- start 2024-04-25T11:43:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e5c1ea740 1 -- start start 2024-04-25T11:43:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e5c1ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e5c1ea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 0x557cf95b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e5c1ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e5c1ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557cf9516c50 con 0x557cf95b1d80 2024-04-25T11:43:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e5c1ea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557cf94b3830 con 0x557cf95b2f50 2024-04-25T11:43:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e5c1ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557cf9512c10 con 0x557cf95b3910 2024-04-25T11:43:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e3bfff640 1 --2- 172.21.15.83:0/3646892107 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1e3c0a3c40 0x7f1e3c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e537fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32844/0 (socket says 172.21.15.83:32844) 2024-04-25T11:43:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e537fe640 1 -- 172.21.15.83:0/2219688441 learned_addr learned my addr 172.21.15.83:0/2219688441 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e537fe640 1 --2- 172.21.15.83:0/3646892107 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1e3c0a3c40 0x7f1e3c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e527fc640 1 --2- 172.21.15.83:0/2219688441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e52ffd640 1 --2- 172.21.15.83:0/2219688441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 0x557cf95b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e3bfff640 1 -- 172.21.15.83:0/3646892107 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f1e480d0190 con 0x7f1e54101e90 2024-04-25T11:43:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e537fe640 1 -- 172.21.15.83:0/2219688441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 msgr2=0x557cf95ba1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e537fe640 1 --2- 172.21.15.83:0/2219688441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95ba1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e537fe640 1 -- 172.21.15.83:0/2219688441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 msgr2=0x557cf95b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e537fe640 1 --2- 172.21.15.83:0/2219688441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 0x557cf95b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e537fe640 1 -- 172.21.15.83:0/2219688441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cf92292d0 con 0x557cf95b1d80 2024-04-25T11:43:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.864+0000 7f1e52ffd640 1 --2- 172.21.15.83:0/2219688441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 0x557cf95b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.865+0000 7f1e537fe640 1 --2- 172.21.15.83:0/2219688441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95b21a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1e44033000 tx=0x7f1e44004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fed6456f534d01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.865+0000 7f1e537fe640 1 --2- 172.21.15.83:0/3646892107 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1e3c0a3c40 0x7f1e3c0a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1e44031500 tx=0x7f1e44046000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.865+0000 7f1e397fa640 1 -- 172.21.15.83:0/2219688441 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44035040 con 0x557cf95b1d80 2024-04-25T11:43:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.865+0000 7f1e397fa640 1 -- 172.21.15.83:0/2219688441 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4404d070 con 0x557cf95b1d80 2024-04-25T11:43:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.865+0000 7f1e397fa640 1 -- 172.21.15.83:0/2219688441 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44003d40 con 0x557cf95b1d80 2024-04-25T11:43:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.865+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/2219688441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 msgr2=0x557cf95b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.865+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/2219688441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95b21a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1e44033000 tx=0x7f1e44004140 comp rx=0 tx=0).stop 2024-04-25T11:43:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.865+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/2219688441 shutdown_connections 2024-04-25T11:43:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.865+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/2219688441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95ba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.865+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/2219688441 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 0x557cf95b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.865+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/2219688441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95b21a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.865+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/2219688441 >> 172.21.15.83:0/2219688441 conn(0x557cf95ad270 msgr2=0x557cf95ad670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.865+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/2219688441 shutdown_connections 2024-04-25T11:43:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.865+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/2219688441 wait complete. 2024-04-25T11:43:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e5c1ea740 1 -- start start 2024-04-25T11:43:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e5c1ea740 1 Processor -- start 2024-04-25T11:43:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e5c1ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95bc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e5c1ea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 0x557cf95bc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e5c1ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95c3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e5c1ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557cf95ba8c0 con 0x557cf95b1d80 2024-04-25T11:43:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e5c1ea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557cf94b1470 con 0x557cf95b2f50 2024-04-25T11:43:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e5c1ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557cf95baa40 con 0x557cf95b3910 2024-04-25T11:43:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e5c1ea740 10 client.0 _ll_register_callbacks cb 0x557cf94d6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e52ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95bc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e52ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95bc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32852/0 (socket says 172.21.15.83:32852) 2024-04-25T11:43:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e52ffd640 1 -- 172.21.15.83:0/693073418 learned_addr learned my addr 172.21.15.83:0/693073418 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:08.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e537fe640 1 --2- 172.21.15.83:0/693073418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e527fc640 1 --2- 172.21.15.83:0/693073418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 0x557cf95bc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e52ffd640 1 -- 172.21.15.83:0/693073418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 msgr2=0x557cf95c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e52ffd640 1 --2- 172.21.15.83:0/693073418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95c3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e52ffd640 1 -- 172.21.15.83:0/693073418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 msgr2=0x557cf95bc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e52ffd640 1 --2- 172.21.15.83:0/693073418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 0x557cf95bc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e52ffd640 1 -- 172.21.15.83:0/693073418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cf92f36b0 con 0x557cf95b1d80 2024-04-25T11:43:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e537fe640 1 --2- 172.21.15.83:0/693073418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e527fc640 1 --2- 172.21.15.83:0/693073418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 0x557cf95bc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e52ffd640 1 --2- 172.21.15.83:0/693073418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95bc1f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c011a90 tx=0x7f1e4c011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678da6c0eb48a941 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.866+0000 7f1e36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.867+0000 7f1e36ff5640 1 -- 172.21.15.83:0/693073418 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c004d20 con 0x557cf95b1d80 2024-04-25T11:43:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.867+0000 7f1e36ff5640 1 -- 172.21.15.83:0/693073418 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f1e4c00fb20 con 0x557cf95b1d80 2024-04-25T11:43:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.867+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/693073418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e4404d070 con 0x557cf95b1d80 2024-04-25T11:43:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.867+0000 7f1e36ff5640 1 -- 172.21.15.83:0/693073418 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1e4c00fd20 con 0x557cf95b1d80 2024-04-25T11:43:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.867+0000 7f1e5c1ea740 10 client.15060 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.867+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/693073418 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557cf95c6c00 con 0x557cf95b1d80 2024-04-25T11:43:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.867+0000 7f1e36ff5640 1 -- 172.21.15.83:0/693073418 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4404d070 con 0x557cf95b1d80 2024-04-25T11:43:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.867+0000 7f1e36ff5640 1 -- 172.21.15.83:0/693073418 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f1e4c0045d0 con 0x557cf95b1d80 2024-04-25T11:43:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.867+0000 7f1e5c1ea740 10 client.15060 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.867+0000 7f1e5c1ea740 10 client.15060 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T11:43:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.867+0000 7f1e5c1ea740 20 client.15060 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.867+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/693073418 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x557cf95c92d0 0x557cf95ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.867+0000 7f1e5c1ea740 4 client.15060 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:43:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.867+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/693073418 --> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cf9524670 con 0x557cf95c92d0 2024-04-25T11:43:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.867+0000 7f1e527fc640 1 --2- 172.21.15.83:0/693073418 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x557cf95c92d0 0x557cf95ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.868+0000 7f1e527fc640 1 --2- 172.21.15.83:0/693073418 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x557cf95c92d0 0x557cf95ca740 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=adab721f6ef7daf0 server_cookie=f9e97371339245cb in_seq=0 out_seq=0 2024-04-25T11:43:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.868+0000 7f1e36ff5640 10 client.15060 ms_handle_connect on v2:172.21.15.111:6834/4167388778 2024-04-25T11:43:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.869+0000 7f1e36ff5640 1 -- 172.21.15.83:0/693073418 <== mds.4 v2:172.21.15.111:6834/4167388778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557cf9524670 con 0x557cf95c92d0 2024-04-25T11:43:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.869+0000 7f1e36ff5640 10 client.15060 handle_command_reply: tid=0 2024-04-25T11:43:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.869+0000 7f1e5c1ea740 1 client.15060 shutdown 2024-04-25T11:43:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.869+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/693073418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 msgr2=0x557cf95bc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.869+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/693073418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95bc1f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c011a90 tx=0x7f1e4c011f50 comp rx=0 tx=0).stop 2024-04-25T11:43:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.869+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/693073418 shutdown_connections 2024-04-25T11:43:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.869+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/693073418 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x557cf95c92d0 0x557cf95ca740 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.869+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/693073418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.869+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/693073418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 0x557cf95bc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.869+0000 7f1e36ff5640 0 client.15060 ms_handle_reset on v2:172.21.15.111:6834/4167388778 2024-04-25T11:43:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.869+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/693073418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95bc1f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.869+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/693073418 >> 172.21.15.83:0/693073418 conn(0x557cf95ad270 msgr2=0x557cf95ac3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:08.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.869+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/693073418 shutdown_connections 2024-04-25T11:43:08.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.869+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/693073418 wait complete. 2024-04-25T11:43:08.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.869+0000 7f1e5c1ea740 20 client.15060 trim_cache size 0 max 16384 2024-04-25T11:43:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e5c1ea740 1 Processor -- start 2024-04-25T11:43:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e5c1ea740 1 -- start start 2024-04-25T11:43:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e5c1ea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b1d80 0x557cf95abb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e5c1ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b2f50 0x557cf95c3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e5c1ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e5c1ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557cf95c8a80 con 0x557cf95b2f50 2024-04-25T11:43:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e5c1ea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557cf95c9150 con 0x557cf95b1d80 2024-04-25T11:43:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e5c1ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557cf95c9820 con 0x557cf95b3910 2024-04-25T11:43:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e527fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b2f50 0x557cf95c3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e527fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b2f50 0x557cf95c3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32866/0 (socket says 172.21.15.83:32866) 2024-04-25T11:43:08.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e527fc640 1 -- 172.21.15.83:0/1148208690 learned_addr learned my addr 172.21.15.83:0/1148208690 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:08.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e52ffd640 1 --2- 172.21.15.83:0/1148208690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b1d80 0x557cf95abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e527fc640 1 -- 172.21.15.83:0/1148208690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 msgr2=0x557cf95ac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:08.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e527fc640 1 --2- 172.21.15.83:0/1148208690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e527fc640 1 -- 172.21.15.83:0/1148208690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b1d80 msgr2=0x557cf95abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:08.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e527fc640 1 --2- 172.21.15.83:0/1148208690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b1d80 0x557cf95abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e527fc640 1 -- 172.21.15.83:0/1148208690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cf95bd320 con 0x557cf95b2f50 2024-04-25T11:43:08.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e52ffd640 1 --2- 172.21.15.83:0/1148208690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b1d80 0x557cf95abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:08.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e527fc640 1 --2- 172.21.15.83:0/1148208690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b2f50 0x557cf95c3730 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1e480cdb80 tx=0x7f1e48007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ff348febad5356 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:08.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.887+0000 7f1e38ff9640 1 -- 172.21.15.83:0/1148208690 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48020040 con 0x557cf95b2f50 2024-04-25T11:43:08.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.888+0000 7f1e38ff9640 1 -- 172.21.15.83:0/1148208690 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e48009de0 con 0x557cf95b2f50 2024-04-25T11:43:08.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.888+0000 7f1e38ff9640 1 -- 172.21.15.83:0/1148208690 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48098990 con 0x557cf95b2f50 2024-04-25T11:43:08.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.888+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/1148208690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b2f50 msgr2=0x557cf95c3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.888+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/1148208690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b2f50 0x557cf95c3730 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1e480cdb80 tx=0x7f1e48007f90 comp rx=0 tx=0).stop 2024-04-25T11:43:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.888+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/1148208690 shutdown_connections 2024-04-25T11:43:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.888+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/1148208690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b2f50 0x557cf95c3730 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.888+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/1148208690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95ac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.888+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/1148208690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b1d80 0x557cf95abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.888+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/1148208690 >> 172.21.15.83:0/1148208690 conn(0x557cf95ad270 msgr2=0x557cf95ad650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.888+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/1148208690 shutdown_connections 2024-04-25T11:43:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.888+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/1148208690 wait complete. 2024-04-25T11:43:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.888+0000 7f1e5c1ea740 1 -- start start 2024-04-25T11:43:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.888+0000 7f1e5c1ea740 1 Processor -- start 2024-04-25T11:43:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e5c1ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95bf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e5c1ea740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 0x557cf95bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e5c1ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95c2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e5c1ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557cf95ca5c0 con 0x557cf95b1d80 2024-04-25T11:43:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e5c1ea740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x557cf95c9ef0 con 0x557cf95b2f50 2024-04-25T11:43:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e5c1ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557cf95cac90 con 0x557cf95b3910 2024-04-25T11:43:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e5c1ea740 10 client.0 _ll_register_callbacks cb 0x7f1e4c00a2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e52ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95bf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e52ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95bf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32876/0 (socket says 172.21.15.83:32876) 2024-04-25T11:43:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e52ffd640 1 -- 172.21.15.83:0/1066361462 learned_addr learned my addr 172.21.15.83:0/1066361462 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e527fc640 1 --2- 172.21.15.83:0/1066361462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 0x557cf95bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e537fe640 1 --2- 172.21.15.83:0/1066361462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e52ffd640 1 -- 172.21.15.83:0/1066361462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 msgr2=0x557cf95c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e52ffd640 1 --2- 172.21.15.83:0/1066361462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95c2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e52ffd640 1 -- 172.21.15.83:0/1066361462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 msgr2=0x557cf95bfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e52ffd640 1 --2- 172.21.15.83:0/1066361462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 0x557cf95bfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e52ffd640 1 -- 172.21.15.83:0/1066361462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cf95c2f20 con 0x557cf95b1d80 2024-04-25T11:43:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e527fc640 1 --2- 172.21.15.83:0/1066361462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 0x557cf95bfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e52ffd640 1 --2- 172.21.15.83:0/1066361462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95bf960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c000f80 tx=0x7f1e4c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ee4bd27bb26263 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e36ff5640 1 -- 172.21.15.83:0/1066361462 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c01cd30 con 0x557cf95b1d80 2024-04-25T11:43:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e36ff5640 1 -- 172.21.15.83:0/1066361462 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f1e4c004a00 con 0x557cf95b1d80 2024-04-25T11:43:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/1066361462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e48009de0 con 0x557cf95b1d80 2024-04-25T11:43:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e36ff5640 1 -- 172.21.15.83:0/1066361462 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1e4c004c40 con 0x557cf95b1d80 2024-04-25T11:43:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e5c1ea740 10 client.15066 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.889+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/1066361462 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x557cf962f7c0 con 0x557cf95b1d80 2024-04-25T11:43:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.890+0000 7f1e36ff5640 1 -- 172.21.15.83:0/1066361462 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e48009de0 con 0x557cf95b1d80 2024-04-25T11:43:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.890+0000 7f1e36ff5640 1 -- 172.21.15.83:0/1066361462 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f1e4c004370 con 0x557cf95b1d80 2024-04-25T11:43:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.890+0000 7f1e5c1ea740 10 client.15066 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.890+0000 7f1e5c1ea740 10 client.15066 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T11:43:08.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.890+0000 7f1e5c1ea740 20 client.15066 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.890+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/1066361462 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x557cf9634370 0x557cf9633080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.890+0000 7f1e5c1ea740 4 client.15066 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:43:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.890+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/1066361462 --> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] -- command(tid 0: {"prefix": "status"}) -- 0x557cf951e4e0 con 0x557cf9634370 2024-04-25T11:43:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.890+0000 7f1e527fc640 1 --2- 172.21.15.83:0/1066361462 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x557cf9634370 0x557cf9633080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.890+0000 7f1e36ff5640 10 client.15066 ms_handle_connect on v2:172.21.15.111:6834/4167388778 2024-04-25T11:43:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.890+0000 7f1e527fc640 1 --2- 172.21.15.83:0/1066361462 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x557cf9634370 0x557cf9633080 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=687f0812b5db8047 server_cookie=4204b82f7f9ed68c in_seq=0 out_seq=0 2024-04-25T11:43:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e36ff5640 1 -- 172.21.15.83:0/1066361462 <== mds.4 v2:172.21.15.111:6834/4167388778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557cf951e4e0 con 0x557cf9634370 2024-04-25T11:43:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e36ff5640 10 client.15066 handle_command_reply: tid=0 2024-04-25T11:43:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e5c1ea740 1 client.15066 shutdown 2024-04-25T11:43:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/1066361462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 msgr2=0x557cf95bf960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/1066361462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95bf960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c000f80 tx=0x7f1e4c01ca00 comp rx=0 tx=0).stop 2024-04-25T11:43:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/1066361462 shutdown_connections 2024-04-25T11:43:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/1066361462 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x557cf9634370 0x557cf9633080 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/1066361462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557cf95b3910 0x557cf95c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/1066361462 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557cf95b2f50 0x557cf95bfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e36ff5640 0 client.15066 ms_handle_reset on v2:172.21.15.111:6834/4167388778 2024-04-25T11:43:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e5c1ea740 1 --2- 172.21.15.83:0/1066361462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557cf95b1d80 0x557cf95bf960 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/1066361462 >> 172.21.15.83:0/1066361462 conn(0x557cf95ad270 msgr2=0x557cf95b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/1066361462 shutdown_connections 2024-04-25T11:43:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e5c1ea740 1 -- 172.21.15.83:0/1066361462 wait complete. 2024-04-25T11:43:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e5c1ea740 20 client.15066 trim_cache size 0 max 16384 2024-04-25T11:43:08.885 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:43:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "7fcf54c6-02f7-11ef-bc93-c7b262605968", 2024-04-25T11:43:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 4, 2024-04-25T11:43:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24430, 2024-04-25T11:43:08.885 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T11:43:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T11:43:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T11:43:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 78.276173666000005, 2024-04-25T11:43:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-04-25T11:43:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-25T11:43:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T11:43:08.886 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 105.47376019799999 2024-04-25T11:43:08.886 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:43:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e53fff640 1 -- 172.21.15.83:0/3646892107 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1e3c0a3c40 msgr2=0x7f1e3c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e53fff640 1 --2- 172.21.15.83:0/3646892107 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1e3c0a3c40 0x7f1e3c0a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1e44031500 tx=0x7f1e44046000 comp rx=0 tx=0).stop 2024-04-25T11:43:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e53fff640 1 -- 172.21.15.83:0/3646892107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e54101e90 msgr2=0x7f1e5419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e53fff640 1 --2- 172.21.15.83:0/3646892107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e54101e90 0x7f1e5419cc70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800d950 tx=0x7f1e4800de10 comp rx=0 tx=0).stop 2024-04-25T11:43:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e53fff640 1 -- 172.21.15.83:0/3646892107 shutdown_connections 2024-04-25T11:43:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e53fff640 1 --2- 172.21.15.83:0/3646892107 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f1e3c0a3c40 0x7f1e3c0a6100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e53fff640 1 --2- 172.21.15.83:0/3646892107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e54101e90 0x7f1e5419cc70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e53fff640 1 --2- 172.21.15.83:0/3646892107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e54102850 0x7f1e5419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e53fff640 1 --2- 172.21.15.83:0/3646892107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e54100ca0 0x7f1e5406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e53fff640 1 -- 172.21.15.83:0/3646892107 >> 172.21.15.83:0/3646892107 conn(0x7f1e540fc230 msgr2=0x7f1e54105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e53fff640 1 -- 172.21.15.83:0/3646892107 shutdown_connections 2024-04-25T11:43:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:08.891+0000 7f1e53fff640 1 -- 172.21.15.83:0/3646892107 wait complete. 2024-04-25T11:43:08.889 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T11:43:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.001+0000 7fcf77213640 1 -- 172.21.15.83:0/1659869684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf701029a0 msgr2=0x7fcf701092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.001+0000 7fcf77213640 1 --2- 172.21.15.83:0/1659869684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf701029a0 0x7fcf701092c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009ca0 tx=0x7fcf6802a8c0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.001+0000 7fcf77213640 1 -- 172.21.15.83:0/1659869684 shutdown_connections 2024-04-25T11:43:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.001+0000 7fcf77213640 1 --2- 172.21.15.83:0/1659869684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf701029a0 0x7fcf701092c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.001+0000 7fcf77213640 1 --2- 172.21.15.83:0/1659869684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf70101fe0 0x7fcf70102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.001+0000 7fcf77213640 1 --2- 172.21.15.83:0/1659869684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf70100e90 0x7fcf701012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.001+0000 7fcf77213640 1 -- 172.21.15.83:0/1659869684 >> 172.21.15.83:0/1659869684 conn(0x7fcf700fc400 msgr2=0x7fcf700fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.002+0000 7fcf77213640 1 -- 172.21.15.83:0/1659869684 shutdown_connections 2024-04-25T11:43:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.002+0000 7fcf77213640 1 -- 172.21.15.83:0/1659869684 wait complete. 2024-04-25T11:43:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.002+0000 7fcf77213640 1 Processor -- start 2024-04-25T11:43:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.002+0000 7fcf77213640 1 -- start start 2024-04-25T11:43:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.002+0000 7fcf77213640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf70100e90 0x7fcf70069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.002+0000 7fcf77213640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf70101fe0 0x7fcf7019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.002+0000 7fcf77213640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf701029a0 0x7fcf701a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.002+0000 7fcf77213640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7010cae0 con 0x7fcf70101fe0 2024-04-25T11:43:08.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.002+0000 7fcf77213640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7010c960 con 0x7fcf701029a0 2024-04-25T11:43:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.002+0000 7fcf77213640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7010cc60 con 0x7fcf70100e90 2024-04-25T11:43:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.002+0000 7fcf6ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf70101fe0 0x7fcf7019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.002+0000 7fcf6ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf70101fe0 0x7fcf7019f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32894/0 (socket says 172.21.15.83:32894) 2024-04-25T11:43:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.002+0000 7fcf6ffff640 1 -- 172.21.15.83:0/1866973642 learned_addr learned my addr 172.21.15.83:0/1866973642 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.002+0000 7fcf74faf640 1 --2- 172.21.15.83:0/1866973642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf70100e90 0x7fcf70069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.002+0000 7fcf757b0640 1 --2- 172.21.15.83:0/1866973642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf701029a0 0x7fcf701a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.003+0000 7fcf6ffff640 1 -- 172.21.15.83:0/1866973642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf70100e90 msgr2=0x7fcf70069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.003+0000 7fcf6ffff640 1 --2- 172.21.15.83:0/1866973642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf70100e90 0x7fcf70069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.003+0000 7fcf6ffff640 1 -- 172.21.15.83:0/1866973642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf701029a0 msgr2=0x7fcf701a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.003+0000 7fcf6ffff640 1 --2- 172.21.15.83:0/1866973642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf701029a0 0x7fcf701a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.003+0000 7fcf6ffff640 1 -- 172.21.15.83:0/1866973642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf701a2180 con 0x7fcf70101fe0 2024-04-25T11:43:08.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.003+0000 7fcf6ffff640 1 --2- 172.21.15.83:0/1866973642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf70101fe0 0x7fcf7019f440 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00b450 tx=0x7fcf5c00b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:08.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.004+0000 7fcf6dffb640 1 -- 172.21.15.83:0/1866973642 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c015020 con 0x7fcf70101fe0 2024-04-25T11:43:08.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.004+0000 7fcf77213640 1 -- 172.21.15.83:0/1866973642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf701a2410 con 0x7fcf70101fe0 2024-04-25T11:43:08.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.004+0000 7fcf6dffb640 1 -- 172.21.15.83:0/1866973642 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf5c011ce0 con 0x7fcf70101fe0 2024-04-25T11:43:08.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.004+0000 7fcf6dffb640 1 -- 172.21.15.83:0/1866973642 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c010dc0 con 0x7fcf70101fe0 2024-04-25T11:43:08.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.004+0000 7fcf77213640 1 -- 172.21.15.83:0/1866973642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf701a29e0 con 0x7fcf70101fe0 2024-04-25T11:43:08.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.007+0000 7fcf6dffb640 1 -- 172.21.15.83:0/1866973642 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcf5c018020 con 0x7fcf70101fe0 2024-04-25T11:43:08.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.007+0000 7fcf7a008740 1 Processor -- start 2024-04-25T11:43:08.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.007+0000 7fcf7a008740 1 -- start start 2024-04-25T11:43:08.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.007+0000 7fcf6dffb640 1 --2- 172.21.15.83:0/1866973642 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fcf600a3c70 0x7fcf600a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.007+0000 7fcf7a008740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5ea9b80 0x5602c5ea9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.007+0000 7fcf7a008740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eaad50 0x5602c5eab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:08.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.007+0000 7fcf7a008740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5eab710 0x5602c5eb1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.007+0000 7fcf7a008740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602c5e0ec50 con 0x5602c5ea9b80 2024-04-25T11:43:09.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.007+0000 7fcf7a008740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5602c5dab830 con 0x5602c5eaad50 2024-04-25T11:43:09.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.007+0000 7fcf7a008740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5602c5e0ac10 con 0x5602c5eab710 2024-04-25T11:43:09.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf74faf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5ea9b80 0x5602c5ea9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf757b0640 1 --2- 172.21.15.83:0/1866973642 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fcf600a3c70 0x7fcf600a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf74faf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5ea9b80 0x5602c5ea9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32896/0 (socket says 172.21.15.83:32896) 2024-04-25T11:43:09.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf74faf640 1 -- 172.21.15.83:0/2860921629 learned_addr learned my addr 172.21.15.83:0/2860921629 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf6dffb640 1 -- 172.21.15.83:0/1866973642 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fcf5c0cfc80 con 0x7fcf70101fe0 2024-04-25T11:43:09.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf757b0640 1 --2- 172.21.15.83:0/2860921629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5eab710 0x5602c5eb1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf6ffff640 1 --2- 172.21.15.83:0/2860921629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eaad50 0x5602c5eab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf74faf640 1 -- 172.21.15.83:0/2860921629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5eab710 msgr2=0x5602c5eb1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf74faf640 1 --2- 172.21.15.83:0/2860921629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5eab710 0x5602c5eb1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf74faf640 1 -- 172.21.15.83:0/2860921629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eaad50 msgr2=0x5602c5eab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf74faf640 1 --2- 172.21.15.83:0/2860921629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eaad50 0x5602c5eab1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf74faf640 1 -- 172.21.15.83:0/2860921629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c5b212d0 con 0x5602c5ea9b80 2024-04-25T11:43:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf757b0640 1 --2- 172.21.15.83:0/1866973642 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fcf600a3c70 0x7fcf600a6130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcf680098d0 tx=0x7fcf680315d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf74faf640 1 --2- 172.21.15.83:0/2860921629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5ea9b80 0x5602c5ea9fa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009a70 tx=0x7fcf64009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3da373ea790ce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf5affd640 1 -- 172.21.15.83:0/2860921629 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64017070 con 0x5602c5ea9b80 2024-04-25T11:43:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf5affd640 1 -- 172.21.15.83:0/2860921629 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf64013470 con 0x5602c5ea9b80 2024-04-25T11:43:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.008+0000 7fcf5affd640 1 -- 172.21.15.83:0/2860921629 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64012610 con 0x5602c5ea9b80 2024-04-25T11:43:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.009+0000 7fcf7a008740 1 -- 172.21.15.83:0/2860921629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5ea9b80 msgr2=0x5602c5ea9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.009+0000 7fcf7a008740 1 --2- 172.21.15.83:0/2860921629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5ea9b80 0x5602c5ea9fa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009a70 tx=0x7fcf64009f30 comp rx=0 tx=0).stop 2024-04-25T11:43:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.009+0000 7fcf7a008740 1 -- 172.21.15.83:0/2860921629 shutdown_connections 2024-04-25T11:43:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.009+0000 7fcf7a008740 1 --2- 172.21.15.83:0/2860921629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5eab710 0x5602c5eb1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.009+0000 7fcf7a008740 1 --2- 172.21.15.83:0/2860921629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eaad50 0x5602c5eab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.009+0000 7fcf7a008740 1 --2- 172.21.15.83:0/2860921629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5ea9b80 0x5602c5ea9fa0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.009+0000 7fcf7a008740 1 -- 172.21.15.83:0/2860921629 >> 172.21.15.83:0/2860921629 conn(0x5602c5ea5070 msgr2=0x5602c5ea5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.009+0000 7fcf7a008740 1 -- 172.21.15.83:0/2860921629 shutdown_connections 2024-04-25T11:43:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.009+0000 7fcf7a008740 1 -- 172.21.15.83:0/2860921629 wait complete. 2024-04-25T11:43:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.009+0000 7fcf7a008740 1 -- start start 2024-04-25T11:43:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.009+0000 7fcf7a008740 1 Processor -- start 2024-04-25T11:43:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf7a008740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5ea9b80 0x5602c5eb3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf7a008740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5eaad50 0x5602c5eb4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf7a008740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eab710 0x5602c5ebb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf7a008740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602c5eb2720 con 0x5602c5eaad50 2024-04-25T11:43:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf7a008740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5602c5da9470 con 0x5602c5eab710 2024-04-25T11:43:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf7a008740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5602c5eb28a0 con 0x5602c5ea9b80 2024-04-25T11:43:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf7a008740 10 client.0 _ll_register_callbacks cb 0x5602c5dcef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf6ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5eaad50 0x5602c5eb4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf6ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5eaad50 0x5602c5eb4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32900/0 (socket says 172.21.15.83:32900) 2024-04-25T11:43:09.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf6ffff640 1 -- 172.21.15.83:0/2074570280 learned_addr learned my addr 172.21.15.83:0/2074570280 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf757b0640 1 --2- 172.21.15.83:0/2074570280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eab710 0x5602c5ebb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf74faf640 1 --2- 172.21.15.83:0/2074570280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5ea9b80 0x5602c5eb3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf6ffff640 1 -- 172.21.15.83:0/2074570280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5ea9b80 msgr2=0x5602c5eb3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf6ffff640 1 --2- 172.21.15.83:0/2074570280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5ea9b80 0x5602c5eb3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf6ffff640 1 -- 172.21.15.83:0/2074570280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eab710 msgr2=0x5602c5ebb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf6ffff640 1 --2- 172.21.15.83:0/2074570280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eab710 0x5602c5ebb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf6ffff640 1 -- 172.21.15.83:0/2074570280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602c5beb6b0 con 0x5602c5eaad50 2024-04-25T11:43:09.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf757b0640 1 --2- 172.21.15.83:0/2074570280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eab710 0x5602c5ebb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:09.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf74faf640 1 --2- 172.21.15.83:0/2074570280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5ea9b80 0x5602c5eb3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:09.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf6ffff640 1 --2- 172.21.15.83:0/2074570280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5eaad50 0x5602c5eb4510 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009fd0 tx=0x7fcf5c09c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40136f37f788d4dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:09.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf4ffff640 1 -- 172.21.15.83:0/2074570280 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c0cd420 con 0x5602c5eaad50 2024-04-25T11:43:09.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf4ffff640 1 -- 172.21.15.83:0/2074570280 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fcf5c0cd890 con 0x5602c5eaad50 2024-04-25T11:43:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf7a008740 1 -- 172.21.15.83:0/2074570280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf64013470 con 0x5602c5eaad50 2024-04-25T11:43:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf4ffff640 1 -- 172.21.15.83:0/2074570280 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcf5c0b5050 con 0x5602c5eaad50 2024-04-25T11:43:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf7a008740 10 client.15078 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.010+0000 7fcf7a008740 1 -- 172.21.15.83:0/2074570280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5602c5ebe930 con 0x5602c5eaad50 2024-04-25T11:43:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.011+0000 7fcf4ffff640 1 -- 172.21.15.83:0/2074570280 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf64013470 con 0x5602c5eaad50 2024-04-25T11:43:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.011+0000 7fcf4ffff640 1 -- 172.21.15.83:0/2074570280 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fcf5c098590 con 0x5602c5eaad50 2024-04-25T11:43:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.011+0000 7fcf7a008740 10 client.15078 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.011+0000 7fcf7a008740 10 client.15078 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T11:43:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.011+0000 7fcf7a008740 20 client.15078 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.011+0000 7fcf7a008740 1 --2- 172.21.15.83:0/2074570280 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5602c5ec0f90 0x5602c5ec2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.011+0000 7fcf7a008740 4 client.15078 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:43:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.011+0000 7fcf7a008740 1 -- 172.21.15.83:0/2074570280 --> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602c5e1c670 con 0x5602c5ec0f90 2024-04-25T11:43:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.011+0000 7fcf74faf640 1 --2- 172.21.15.83:0/2074570280 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5602c5ec0f90 0x5602c5ec2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.011+0000 7fcf4ffff640 10 client.15078 ms_handle_connect on v2:172.21.15.94:6836/2475976592 2024-04-25T11:43:09.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.011+0000 7fcf74faf640 1 --2- 172.21.15.83:0/2074570280 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5602c5ec0f90 0x5602c5ec2400 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e23cb25474d0427a server_cookie=1f74769030569930 in_seq=0 out_seq=0 2024-04-25T11:43:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.012+0000 7fcf4ffff640 1 -- 172.21.15.83:0/2074570280 <== mds.3 v2:172.21.15.94:6836/2475976592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602c5e1c670 con 0x5602c5ec0f90 2024-04-25T11:43:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.012+0000 7fcf4ffff640 10 client.15078 handle_command_reply: tid=0 2024-04-25T11:43:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.012+0000 7fcf7a008740 1 client.15078 shutdown 2024-04-25T11:43:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.012+0000 7fcf7a008740 1 -- 172.21.15.83:0/2074570280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5eaad50 msgr2=0x5602c5eb4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.012+0000 7fcf7a008740 1 --2- 172.21.15.83:0/2074570280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5eaad50 0x5602c5eb4510 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009fd0 tx=0x7fcf5c09c760 comp rx=0 tx=0).stop 2024-04-25T11:43:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.012+0000 7fcf7a008740 1 -- 172.21.15.83:0/2074570280 shutdown_connections 2024-04-25T11:43:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.012+0000 7fcf7a008740 1 --2- 172.21.15.83:0/2074570280 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5602c5ec0f90 0x5602c5ec2400 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.012+0000 7fcf7a008740 1 --2- 172.21.15.83:0/2074570280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5eaad50 0x5602c5eb4510 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.012+0000 7fcf4ffff640 0 client.15078 ms_handle_reset on v2:172.21.15.94:6836/2475976592 2024-04-25T11:43:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.012+0000 7fcf7a008740 1 --2- 172.21.15.83:0/2074570280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eab710 0x5602c5ebb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.012+0000 7fcf7a008740 1 --2- 172.21.15.83:0/2074570280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5ea9b80 0x5602c5eb3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.012+0000 7fcf7a008740 1 -- 172.21.15.83:0/2074570280 >> 172.21.15.83:0/2074570280 conn(0x5602c5ea5070 msgr2=0x5602c5ea41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.013+0000 7fcf7a008740 1 -- 172.21.15.83:0/2074570280 shutdown_connections 2024-04-25T11:43:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.013+0000 7fcf7a008740 1 -- 172.21.15.83:0/2074570280 wait complete. 2024-04-25T11:43:09.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.013+0000 7fcf7a008740 20 client.15078 trim_cache size 0 max 16384 2024-04-25T11:43:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.030+0000 7fcf7a008740 1 Processor -- start 2024-04-25T11:43:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.030+0000 7fcf7a008740 1 -- start start 2024-04-25T11:43:09.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf7a008740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5ea9b80 0x5602c5eb3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf7a008740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eaad50 0x5602c5ebb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf7a008740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5eab710 0x5602c5eb4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf7a008740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602c5ec0740 con 0x5602c5ea9b80 2024-04-25T11:43:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf7a008740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5602c5ec0e10 con 0x5602c5eaad50 2024-04-25T11:43:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf7a008740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5602c5ec14e0 con 0x5602c5eab710 2024-04-25T11:43:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf74faf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5ea9b80 0x5602c5eb3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf74faf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5ea9b80 0x5602c5eb3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32904/0 (socket says 172.21.15.83:32904) 2024-04-25T11:43:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf74faf640 1 -- 172.21.15.83:0/1935917460 learned_addr learned my addr 172.21.15.83:0/1935917460 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf74faf640 1 -- 172.21.15.83:0/1935917460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5eab710 msgr2=0x5602c5eb4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf74faf640 1 --2- 172.21.15.83:0/1935917460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5eab710 0x5602c5eb4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf74faf640 1 -- 172.21.15.83:0/1935917460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eaad50 msgr2=0x5602c5ebb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf6ffff640 1 --2- 172.21.15.83:0/1935917460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eaad50 0x5602c5ebb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf74faf640 1 --2- 172.21.15.83:0/1935917460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eaad50 0x5602c5ebb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf74faf640 1 -- 172.21.15.83:0/1935917460 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c5ea8f90 con 0x5602c5ea9b80 2024-04-25T11:43:09.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf6ffff640 1 --2- 172.21.15.83:0/1935917460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eaad50 0x5602c5ebb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf74faf640 1 --2- 172.21.15.83:0/1935917460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5ea9b80 0x5602c5eb3be0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcf640091f0 tx=0x7fcf64009180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8edf1727758d19a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf5a7fc640 1 -- 172.21.15.83:0/1935917460 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64017040 con 0x5602c5ea9b80 2024-04-25T11:43:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf5a7fc640 1 -- 172.21.15.83:0/1935917460 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf6402a070 con 0x5602c5ea9b80 2024-04-25T11:43:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.031+0000 7fcf5a7fc640 1 -- 172.21.15.83:0/1935917460 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64013a50 con 0x5602c5ea9b80 2024-04-25T11:43:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.032+0000 7fcf7a008740 1 -- 172.21.15.83:0/1935917460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5ea9b80 msgr2=0x5602c5eb3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.032+0000 7fcf7a008740 1 --2- 172.21.15.83:0/1935917460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5ea9b80 0x5602c5eb3be0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcf640091f0 tx=0x7fcf64009180 comp rx=0 tx=0).stop 2024-04-25T11:43:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.032+0000 7fcf7a008740 1 -- 172.21.15.83:0/1935917460 shutdown_connections 2024-04-25T11:43:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.032+0000 7fcf7a008740 1 --2- 172.21.15.83:0/1935917460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5eab710 0x5602c5eb4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.032+0000 7fcf7a008740 1 --2- 172.21.15.83:0/1935917460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5eaad50 0x5602c5ebb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.032+0000 7fcf7a008740 1 --2- 172.21.15.83:0/1935917460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5ea9b80 0x5602c5eb3be0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.032+0000 7fcf7a008740 1 -- 172.21.15.83:0/1935917460 >> 172.21.15.83:0/1935917460 conn(0x5602c5ea5070 msgr2=0x5602c5eae7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.032+0000 7fcf7a008740 1 -- 172.21.15.83:0/1935917460 shutdown_connections 2024-04-25T11:43:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.032+0000 7fcf7a008740 1 -- 172.21.15.83:0/1935917460 wait complete. 2024-04-25T11:43:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.032+0000 7fcf7a008740 1 -- start start 2024-04-25T11:43:09.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.032+0000 7fcf7a008740 1 Processor -- start 2024-04-25T11:43:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf7a008740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5ea9b80 0x5602c5eb7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf7a008740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5eaad50 0x5602c5eb78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf7a008740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5eab710 0x5602c5eba8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf7a008740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5602c5ec2280 con 0x5602c5eaad50 2024-04-25T11:43:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf7a008740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5602c5ec1bb0 con 0x5602c5ea9b80 2024-04-25T11:43:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf7a008740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5602c5ec2950 con 0x5602c5eab710 2024-04-25T11:43:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf7a008740 10 client.0 _ll_register_callbacks cb 0x7fcf5c0981c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf6ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5eaad50 0x5602c5eb78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf6ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5eaad50 0x5602c5eb78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32920/0 (socket says 172.21.15.83:32920) 2024-04-25T11:43:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf6ffff640 1 -- 172.21.15.83:0/3377258244 learned_addr learned my addr 172.21.15.83:0/3377258244 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf74faf640 1 --2- 172.21.15.83:0/3377258244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5ea9b80 0x5602c5eb7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf757b0640 1 --2- 172.21.15.83:0/3377258244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5eab710 0x5602c5eba8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf6ffff640 1 -- 172.21.15.83:0/3377258244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5eab710 msgr2=0x5602c5eba8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf6ffff640 1 --2- 172.21.15.83:0/3377258244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5eab710 0x5602c5eba8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf6ffff640 1 -- 172.21.15.83:0/3377258244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5ea9b80 msgr2=0x5602c5eb7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf6ffff640 1 --2- 172.21.15.83:0/3377258244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5ea9b80 0x5602c5eb7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf6ffff640 1 -- 172.21.15.83:0/3377258244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602c5ebae90 con 0x5602c5eaad50 2024-04-25T11:43:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf757b0640 1 --2- 172.21.15.83:0/3377258244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5eab710 0x5602c5eba8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf6ffff640 1 --2- 172.21.15.83:0/3377258244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5eaad50 0x5602c5eb78d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00bd50 tx=0x7fcf5c09bfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f468aaa835bb34b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf4ffff640 1 -- 172.21.15.83:0/3377258244 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c0ba060 con 0x5602c5eaad50 2024-04-25T11:43:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf4ffff640 1 -- 172.21.15.83:0/3377258244 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fcf5c0b5380 con 0x5602c5eaad50 2024-04-25T11:43:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf7a008740 1 -- 172.21.15.83:0/3377258244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf6402a070 con 0x5602c5eaad50 2024-04-25T11:43:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf4ffff640 1 -- 172.21.15.83:0/3377258244 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcf5c0c5050 con 0x5602c5eaad50 2024-04-25T11:43:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf7a008740 10 client.15084 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.033+0000 7fcf7a008740 1 -- 172.21.15.83:0/3377258244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5602c5f27660 con 0x5602c5eaad50 2024-04-25T11:43:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.034+0000 7fcf4ffff640 1 -- 172.21.15.83:0/3377258244 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf6402a070 con 0x5602c5eaad50 2024-04-25T11:43:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.034+0000 7fcf4ffff640 1 -- 172.21.15.83:0/3377258244 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fcf5c09b3f0 con 0x5602c5eaad50 2024-04-25T11:43:09.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.034+0000 7fcf7a008740 10 client.15084 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.034+0000 7fcf7a008740 10 client.15084 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T11:43:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.034+0000 7fcf7a008740 20 client.15084 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.034+0000 7fcf7a008740 1 --2- 172.21.15.83:0/3377258244 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5602c5f2c2a0 0x5602c5f2afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.034+0000 7fcf7a008740 4 client.15084 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:43:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.034+0000 7fcf7a008740 1 -- 172.21.15.83:0/3377258244 --> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] -- command(tid 0: {"prefix": "status"}) -- 0x5602c5e164e0 con 0x5602c5f2c2a0 2024-04-25T11:43:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.034+0000 7fcf74faf640 1 --2- 172.21.15.83:0/3377258244 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5602c5f2c2a0 0x5602c5f2afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.034+0000 7fcf74faf640 1 --2- 172.21.15.83:0/3377258244 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5602c5f2c2a0 0x5602c5f2afb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c3d113179737e4a4 server_cookie=e2cd68ca2c03632b in_seq=0 out_seq=0 2024-04-25T11:43:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.034+0000 7fcf4ffff640 10 client.15084 ms_handle_connect on v2:172.21.15.94:6836/2475976592 2024-04-25T11:43:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf4ffff640 1 -- 172.21.15.83:0/3377258244 <== mds.3 v2:172.21.15.94:6836/2475976592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5602c5e164e0 con 0x5602c5f2c2a0 2024-04-25T11:43:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf4ffff640 10 client.15084 handle_command_reply: tid=0 2024-04-25T11:43:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf7a008740 1 client.15084 shutdown 2024-04-25T11:43:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf7a008740 1 -- 172.21.15.83:0/3377258244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5eaad50 msgr2=0x5602c5eb78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf7a008740 1 --2- 172.21.15.83:0/3377258244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5eaad50 0x5602c5eb78d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00bd50 tx=0x7fcf5c09bfe0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf7a008740 1 -- 172.21.15.83:0/3377258244 shutdown_connections 2024-04-25T11:43:09.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf7a008740 1 --2- 172.21.15.83:0/3377258244 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5602c5f2c2a0 0x5602c5f2afb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf7a008740 1 --2- 172.21.15.83:0/3377258244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5602c5eaad50 0x5602c5eb78d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf7a008740 1 --2- 172.21.15.83:0/3377258244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5602c5eab710 0x5602c5eba8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf7a008740 1 --2- 172.21.15.83:0/3377258244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5602c5ea9b80 0x5602c5eb7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf4ffff640 0 client.15084 ms_handle_reset on v2:172.21.15.94:6836/2475976592 2024-04-25T11:43:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf7a008740 1 -- 172.21.15.83:0/3377258244 >> 172.21.15.83:0/3377258244 conn(0x5602c5ea5070 msgr2=0x5602c5eb3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf7a008740 1 -- 172.21.15.83:0/3377258244 shutdown_connections 2024-04-25T11:43:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf7a008740 1 -- 172.21.15.83:0/3377258244 wait complete. 2024-04-25T11:43:09.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf7a008740 20 client.15084 trim_cache size 0 max 16384 2024-04-25T11:43:09.028 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:43:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "7fcf54c6-02f7-11ef-bc93-c7b262605968", 2024-04-25T11:43:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 3, 2024-04-25T11:43:09.028 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24440, 2024-04-25T11:43:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T11:43:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T11:43:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T11:43:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 79.426253044999996, 2024-04-25T11:43:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-04-25T11:43:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-25T11:43:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T11:43:09.029 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 100.052426873 2024-04-25T11:43:09.029 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:43:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf77213640 1 -- 172.21.15.83:0/1866973642 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fcf600a3c70 msgr2=0x7fcf600a6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf77213640 1 --2- 172.21.15.83:0/1866973642 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fcf600a3c70 0x7fcf600a6130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcf680098d0 tx=0x7fcf680315d0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf77213640 1 -- 172.21.15.83:0/1866973642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf70101fe0 msgr2=0x7fcf7019f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf77213640 1 --2- 172.21.15.83:0/1866973642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf70101fe0 0x7fcf7019f440 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00b450 tx=0x7fcf5c00b850 comp rx=0 tx=0).stop 2024-04-25T11:43:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf77213640 1 -- 172.21.15.83:0/1866973642 shutdown_connections 2024-04-25T11:43:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf77213640 1 --2- 172.21.15.83:0/1866973642 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fcf600a3c70 0x7fcf600a6130 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf77213640 1 --2- 172.21.15.83:0/1866973642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf70101fe0 0x7fcf7019f440 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf77213640 1 --2- 172.21.15.83:0/1866973642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf701029a0 0x7fcf701a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf77213640 1 --2- 172.21.15.83:0/1866973642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf70100e90 0x7fcf70069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.035+0000 7fcf77213640 1 -- 172.21.15.83:0/1866973642 >> 172.21.15.83:0/1866973642 conn(0x7fcf700fc400 msgr2=0x7fcf701053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.036+0000 7fcf77213640 1 -- 172.21.15.83:0/1866973642 shutdown_connections 2024-04-25T11:43:09.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.036+0000 7fcf77213640 1 -- 172.21.15.83:0/1866973642 wait complete. 2024-04-25T11:43:09.032 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:09 smithi083 ceph-mon[29330]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:43:09.032 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:09 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1613964539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:43:09.032 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:09 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2787721898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:09.032 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:09 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/536464712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:43:09.033 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T11:43:09.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.144+0000 7f9dbb8a7640 1 -- 172.21.15.83:0/220364527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4102a40 msgr2=0x7f9db4109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.144+0000 7f9dbb8a7640 1 --2- 172.21.15.83:0/220364527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4102a40 0x7f9db4109360 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009ca0 tx=0x7f9dac02a8c0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.144+0000 7f9dbb8a7640 1 -- 172.21.15.83:0/220364527 shutdown_connections 2024-04-25T11:43:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.144+0000 7f9dbb8a7640 1 --2- 172.21.15.83:0/220364527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4102a40 0x7f9db4109360 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.144+0000 7f9dbb8a7640 1 --2- 172.21.15.83:0/220364527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9db4102080 0x7f9db4102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.144+0000 7f9dbb8a7640 1 --2- 172.21.15.83:0/220364527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9db4100f30 0x7f9db4101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.144+0000 7f9dbb8a7640 1 -- 172.21.15.83:0/220364527 >> 172.21.15.83:0/220364527 conn(0x7f9db40fc4a0 msgr2=0x7f9db40fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.144+0000 7f9dbb8a7640 1 -- 172.21.15.83:0/220364527 shutdown_connections 2024-04-25T11:43:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.144+0000 7f9dbb8a7640 1 -- 172.21.15.83:0/220364527 wait complete. 2024-04-25T11:43:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9dbb8a7640 1 Processor -- start 2024-04-25T11:43:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9dbb8a7640 1 -- start start 2024-04-25T11:43:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9dbb8a7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4100f30 0x7f9db406b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9dbb8a7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9db4102080 0x7f9db419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9dbb8a7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9db4102a40 0x7f9db419f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9dbb8a7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9db410ca90 con 0x7f9db4100f30 2024-04-25T11:43:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9dbb8a7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9db410c910 con 0x7f9db4102080 2024-04-25T11:43:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9dbb8a7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9db410cc10 con 0x7f9db4102a40 2024-04-25T11:43:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9db9643640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4100f30 0x7f9db406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9db9643640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4100f30 0x7f9db406b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32934/0 (socket says 172.21.15.83:32934) 2024-04-25T11:43:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9db9643640 1 -- 172.21.15.83:0/4011795552 learned_addr learned my addr 172.21.15.83:0/4011795552 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9db9e44640 1 --2- 172.21.15.83:0/4011795552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9db4102a40 0x7f9db419f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9db8e42640 1 --2- 172.21.15.83:0/4011795552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9db4102080 0x7f9db419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9db9643640 1 -- 172.21.15.83:0/4011795552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9db4102a40 msgr2=0x7f9db419f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9db9643640 1 --2- 172.21.15.83:0/4011795552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9db4102a40 0x7f9db419f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9db9643640 1 -- 172.21.15.83:0/4011795552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9db4102080 msgr2=0x7f9db419cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9db9643640 1 --2- 172.21.15.83:0/4011795552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9db4102080 0x7f9db419cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9db9643640 1 -- 172.21.15.83:0/4011795552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db419fe10 con 0x7f9db4100f30 2024-04-25T11:43:09.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.145+0000 7f9db8e42640 1 --2- 172.21.15.83:0/4011795552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9db4102080 0x7f9db419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:09.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.146+0000 7f9db9643640 1 --2- 172.21.15.83:0/4011795552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4100f30 0x7f9db406b120 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9db000d770 tx=0x7f9db000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.146+0000 7f9daa7fc640 1 -- 172.21.15.83:0/4011795552 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0017070 con 0x7f9db4100f30 2024-04-25T11:43:09.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.146+0000 7f9daa7fc640 1 -- 172.21.15.83:0/4011795552 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9db00054d0 con 0x7f9db4100f30 2024-04-25T11:43:09.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.146+0000 7f9dbb8a7640 1 -- 172.21.15.83:0/4011795552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db41a0100 con 0x7f9db4100f30 2024-04-25T11:43:09.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.146+0000 7f9daa7fc640 1 -- 172.21.15.83:0/4011795552 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0004c30 con 0x7f9db4100f30 2024-04-25T11:43:09.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.146+0000 7f9dbb8a7640 1 -- 172.21.15.83:0/4011795552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db41a0600 con 0x7f9db4100f30 2024-04-25T11:43:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.148+0000 7f9daa7fc640 1 -- 172.21.15.83:0/4011795552 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9db0004e10 con 0x7f9db4100f30 2024-04-25T11:43:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.148+0000 7f9dbe5be740 1 Processor -- start 2024-04-25T11:43:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.148+0000 7f9dbe5be740 1 -- start start 2024-04-25T11:43:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9dbe5be740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b4489f90 0x5605b448a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9dbe5be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b448b160 0x5605b448b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9dbe5be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 0x5605b44923b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9dbe5be740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5605b43eac50 con 0x5605b448b160 2024-04-25T11:43:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9dbe5be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5605b4387830 con 0x5605b448bb20 2024-04-25T11:43:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9dbe5be740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5605b43e6c10 con 0x5605b4489f90 2024-04-25T11:43:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9daa7fc640 1 --2- 172.21.15.83:0/4011795552 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9da00a3c40 0x7f9da00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9db9e44640 1 --2- 172.21.15.83:0/4011795552 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9da00a3c40 0x7f9da00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9db9e44640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b4489f90 0x5605b448a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9db9e44640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b4489f90 0x5605b448a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.83:37702/0 (socket says 172.21.15.83:37702) 2024-04-25T11:43:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9db9e44640 1 -- 172.21.15.83:0/1133455222 learned_addr learned my addr 172.21.15.83:0/1133455222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9db9643640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b448b160 0x5605b448b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9db8e42640 1 --2- 172.21.15.83:0/1133455222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 0x5605b44923b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9daa7fc640 1 -- 172.21.15.83:0/4011795552 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f9db00cffa0 con 0x7f9db4100f30 2024-04-25T11:43:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9db9e44640 1 -- 172.21.15.83:0/1133455222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 msgr2=0x5605b44923b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9db9e44640 1 --2- 172.21.15.83:0/1133455222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 0x5605b44923b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9db9e44640 1 -- 172.21.15.83:0/1133455222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b448b160 msgr2=0x5605b448b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9db9e44640 1 --2- 172.21.15.83:0/1133455222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b448b160 0x5605b448b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9db9e44640 1 -- 172.21.15.83:0/1133455222 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b40fd2d0 con 0x5605b4489f90 2024-04-25T11:43:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9db9e44640 1 --2- 172.21.15.83:0/1133455222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b4489f90 0x5605b448a3b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009430 tx=0x7f9dac031690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3a0b715aa28b8c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.149+0000 7f9db9e44640 1 --2- 172.21.15.83:0/4011795552 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9da00a3c40 0x7f9da00a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9dac02ade0 tx=0x7f9dac036000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.150+0000 7f9d9f7fe640 1 -- 172.21.15.83:0/1133455222 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac04e030 con 0x5605b4489f90 2024-04-25T11:43:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.150+0000 7f9d9f7fe640 1 -- 172.21.15.83:0/1133455222 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dac036070 con 0x5605b4489f90 2024-04-25T11:43:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.150+0000 7f9d9f7fe640 1 -- 172.21.15.83:0/1133455222 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac045040 con 0x5605b4489f90 2024-04-25T11:43:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.150+0000 7f9dbe5be740 1 -- 172.21.15.83:0/1133455222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b4489f90 msgr2=0x5605b448a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.150+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/1133455222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b4489f90 0x5605b448a3b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009430 tx=0x7f9dac031690 comp rx=0 tx=0).stop 2024-04-25T11:43:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.150+0000 7f9dbe5be740 1 -- 172.21.15.83:0/1133455222 shutdown_connections 2024-04-25T11:43:09.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.150+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/1133455222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b448b160 0x5605b448b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.150+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/1133455222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 0x5605b44923b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.150+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/1133455222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b4489f90 0x5605b448a3b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.150+0000 7f9dbe5be740 1 -- 172.21.15.83:0/1133455222 >> 172.21.15.83:0/1133455222 conn(0x5605b4481070 msgr2=0x5605b4481470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.150+0000 7f9dbe5be740 1 -- 172.21.15.83:0/1133455222 shutdown_connections 2024-04-25T11:43:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.150+0000 7f9dbe5be740 1 -- 172.21.15.83:0/1133455222 wait complete. 2024-04-25T11:43:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9dbe5be740 1 -- start start 2024-04-25T11:43:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9dbe5be740 1 Processor -- start 2024-04-25T11:43:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9dbe5be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b4489f90 0x5605b4494420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9dbe5be740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b448b160 0x5605b4494960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9dbe5be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 0x5605b449b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9dbe5be740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5605b4492ad0 con 0x5605b4489f90 2024-04-25T11:43:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9dbe5be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5605b4385470 con 0x5605b448bb20 2024-04-25T11:43:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9dbe5be740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5605b4492c50 con 0x5605b448b160 2024-04-25T11:43:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9dbe5be740 10 client.0 _ll_register_callbacks cb 0x5605b43aaf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:09.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9db9643640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b4489f90 0x5605b4494420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9db9643640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b4489f90 0x5605b4494420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32956/0 (socket says 172.21.15.83:32956) 2024-04-25T11:43:09.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9db9643640 1 -- 172.21.15.83:0/3606314359 learned_addr learned my addr 172.21.15.83:0/3606314359 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9db9e44640 1 --2- 172.21.15.83:0/3606314359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 0x5605b449b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9db8e42640 1 --2- 172.21.15.83:0/3606314359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b448b160 0x5605b4494960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9db9643640 1 -- 172.21.15.83:0/3606314359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b448b160 msgr2=0x5605b4494960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9db9643640 1 --2- 172.21.15.83:0/3606314359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b448b160 0x5605b4494960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9db9643640 1 -- 172.21.15.83:0/3606314359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 msgr2=0x5605b449b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9db9643640 1 --2- 172.21.15.83:0/3606314359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 0x5605b449b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.151+0000 7f9db9643640 1 -- 172.21.15.83:0/3606314359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b41c76b0 con 0x5605b4489f90 2024-04-25T11:43:09.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9db9643640 1 --2- 172.21.15.83:0/3606314359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b4489f90 0x5605b4494420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009f40 tx=0x7f9db009bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c8f1c5c7d09e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9d9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:09.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9d9cff9640 1 -- 172.21.15.83:0/3606314359 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0020040 con 0x5605b4489f90 2024-04-25T11:43:09.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9d9cff9640 1 -- 172.21.15.83:0/3606314359 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f9db0098840 con 0x5605b4489f90 2024-04-25T11:43:09.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3606314359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dac036070 con 0x5605b4489f90 2024-04-25T11:43:09.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9d9cff9640 1 -- 172.21.15.83:0/3606314359 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9db009c8d0 con 0x5605b4489f90 2024-04-25T11:43:09.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9dbe5be740 10 client.15096 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:09.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3606314359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5605b449ed80 con 0x5605b4489f90 2024-04-25T11:43:09.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9d9cff9640 1 -- 172.21.15.83:0/3606314359 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dac036070 con 0x5605b4489f90 2024-04-25T11:43:09.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9d9cff9640 1 -- 172.21.15.83:0/3606314359 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f9db00b2230 con 0x5605b4489f90 2024-04-25T11:43:09.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9dbe5be740 10 client.15096 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:09.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9dbe5be740 10 client.15096 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:43:09.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9dbe5be740 20 client.15096 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:09.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/3606314359 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5605b44a13a0 0x5605b44a2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9dbe5be740 4 client.15096 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:43:09.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3606314359 --> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605b43f8670 con 0x5605b44a13a0 2024-04-25T11:43:09.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.152+0000 7f9db8e42640 1 --2- 172.21.15.83:0/3606314359 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5605b44a13a0 0x5605b44a2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.153+0000 7f9d9cff9640 10 client.15096 ms_handle_connect on v2:172.21.15.111:6836/2927430878 2024-04-25T11:43:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.153+0000 7f9db8e42640 1 --2- 172.21.15.83:0/3606314359 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5605b44a13a0 0x5605b44a2810 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab53e1c7f2cb58be server_cookie=c7e45d07a3a11832 in_seq=0 out_seq=0 2024-04-25T11:43:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.154+0000 7f9d9cff9640 1 -- 172.21.15.83:0/3606314359 <== mds.2 v2:172.21.15.111:6836/2927430878 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605b43f8670 con 0x5605b44a13a0 2024-04-25T11:43:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.154+0000 7f9d9cff9640 10 client.15096 handle_command_reply: tid=0 2024-04-25T11:43:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.154+0000 7f9dbe5be740 1 client.15096 shutdown 2024-04-25T11:43:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.154+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3606314359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b4489f90 msgr2=0x5605b4494420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.154+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/3606314359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b4489f90 0x5605b4494420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009f40 tx=0x7f9db009bc60 comp rx=0 tx=0).stop 2024-04-25T11:43:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.154+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3606314359 shutdown_connections 2024-04-25T11:43:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.154+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/3606314359 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5605b44a13a0 0x5605b44a2810 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.154+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/3606314359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 0x5605b449b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.154+0000 7f9d9cff9640 0 client.15096 ms_handle_reset on v2:172.21.15.111:6836/2927430878 2024-04-25T11:43:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.154+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/3606314359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b448b160 0x5605b4494960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.154+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/3606314359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b4489f90 0x5605b4494420 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.154+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3606314359 >> 172.21.15.83:0/3606314359 conn(0x5605b4481070 msgr2=0x5605b44801d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.154+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3606314359 shutdown_connections 2024-04-25T11:43:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.154+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3606314359 wait complete. 2024-04-25T11:43:09.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.154+0000 7f9dbe5be740 20 client.15096 trim_cache size 0 max 16384 2024-04-25T11:43:09.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9dbe5be740 1 Processor -- start 2024-04-25T11:43:09.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9dbe5be740 1 -- start start 2024-04-25T11:43:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9dbe5be740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b4489f90 0x5605b448b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9dbe5be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b448b160 0x5605b449b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9dbe5be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 0x5605b4494740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9dbe5be740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5605b44a0b50 con 0x5605b448b160 2024-04-25T11:43:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9dbe5be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5605b44a1220 con 0x5605b448bb20 2024-04-25T11:43:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9dbe5be740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5605b44a18f0 con 0x5605b4489f90 2024-04-25T11:43:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9db8e42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b448b160 0x5605b449b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9db8e42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b448b160 0x5605b449b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32960/0 (socket says 172.21.15.83:32960) 2024-04-25T11:43:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9db8e42640 1 -- 172.21.15.83:0/1792482901 learned_addr learned my addr 172.21.15.83:0/1792482901 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9db8e42640 1 -- 172.21.15.83:0/1792482901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b4489f90 msgr2=0x5605b448b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9db8e42640 1 --2- 172.21.15.83:0/1792482901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b4489f90 0x5605b448b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9db8e42640 1 -- 172.21.15.83:0/1792482901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 msgr2=0x5605b4494740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9db8e42640 1 --2- 172.21.15.83:0/1792482901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 0x5605b4494740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9db8e42640 1 -- 172.21.15.83:0/1792482901 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b448abe0 con 0x5605b448b160 2024-04-25T11:43:09.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9db8e42640 1 --2- 172.21.15.83:0/1792482901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b448b160 0x5605b449b930 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9db406ae90 tx=0x7f9da400b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f5218c7d084afe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9d9effd640 1 -- 172.21.15.83:0/1792482901 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da40142d0 con 0x5605b448b160 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9d9effd640 1 -- 172.21.15.83:0/1792482901 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9da401b330 con 0x5605b448b160 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.172+0000 7f9d9effd640 1 -- 172.21.15.83:0/1792482901 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da40128b0 con 0x5605b448b160 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.173+0000 7f9dbe5be740 1 -- 172.21.15.83:0/1792482901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b448b160 msgr2=0x5605b449b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.173+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/1792482901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b448b160 0x5605b449b930 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9db406ae90 tx=0x7f9da400b5a0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.173+0000 7f9dbe5be740 1 -- 172.21.15.83:0/1792482901 shutdown_connections 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.173+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/1792482901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b448b160 0x5605b449b930 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.173+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/1792482901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 0x5605b4494740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.173+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/1792482901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b4489f90 0x5605b448b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.173+0000 7f9dbe5be740 1 -- 172.21.15.83:0/1792482901 >> 172.21.15.83:0/1792482901 conn(0x5605b4481070 msgr2=0x5605b4494030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.173+0000 7f9dbe5be740 1 -- 172.21.15.83:0/1792482901 shutdown_connections 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.173+0000 7f9dbe5be740 1 -- 172.21.15.83:0/1792482901 wait complete. 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.173+0000 7f9dbe5be740 1 -- start start 2024-04-25T11:43:09.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.173+0000 7f9dbe5be740 1 Processor -- start 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9dbe5be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b4489f90 0x5605b4497b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9dbe5be740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b448b160 0x5605b4498080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9dbe5be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 0x5605b449ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9dbe5be740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5605b44a2690 con 0x5605b4489f90 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9dbe5be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5605b44a1fc0 con 0x5605b448bb20 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9dbe5be740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5605b44a2d60 con 0x5605b448b160 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9dbe5be740 10 client.0 _ll_register_callbacks cb 0x7f9db000de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9db9643640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b4489f90 0x5605b4497b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9db9643640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b4489f90 0x5605b4497b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32962/0 (socket says 172.21.15.83:32962) 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9db9643640 1 -- 172.21.15.83:0/3018419092 learned_addr learned my addr 172.21.15.83:0/3018419092 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9db9e44640 1 --2- 172.21.15.83:0/3018419092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 0x5605b449ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9db8e42640 1 --2- 172.21.15.83:0/3018419092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b448b160 0x5605b4498080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9db9643640 1 -- 172.21.15.83:0/3018419092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b448b160 msgr2=0x5605b4498080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9db9643640 1 --2- 172.21.15.83:0/3018419092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b448b160 0x5605b4498080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9db9643640 1 -- 172.21.15.83:0/3018419092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 msgr2=0x5605b449ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9db9643640 1 --2- 172.21.15.83:0/3018419092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 0x5605b449ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9db9643640 1 -- 172.21.15.83:0/3018419092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b449b320 con 0x5605b4489f90 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9db8e42640 1 --2- 172.21.15.83:0/3018419092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b448b160 0x5605b4498080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9db9643640 1 --2- 172.21.15.83:0/3018419092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b4489f90 0x5605b4497b40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9db0098630 tx=0x7f9db00cde80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9947ad917c8c0954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9d9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9d9cff9640 1 -- 172.21.15.83:0/3018419092 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0020070 con 0x5605b4489f90 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9d9cff9640 1 -- 172.21.15.83:0/3018419092 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f9db0098840 con 0x5605b4489f90 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.174+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3018419092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9da401b330 con 0x5605b4489f90 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.175+0000 7f9d9cff9640 1 -- 172.21.15.83:0/3018419092 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9db009bd10 con 0x5605b4489f90 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.175+0000 7f9dbe5be740 10 client.15102 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.175+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3018419092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5605b4507b80 con 0x5605b4489f90 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.175+0000 7f9d9cff9640 1 -- 172.21.15.83:0/3018419092 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9da401b330 con 0x5605b4489f90 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.175+0000 7f9d9cff9640 1 -- 172.21.15.83:0/3018419092 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f9db0085360 con 0x5605b4489f90 2024-04-25T11:43:09.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.175+0000 7f9dbe5be740 10 client.15102 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.175+0000 7f9dbe5be740 10 client.15102 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.175+0000 7f9dbe5be740 20 client.15102 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.175+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/3018419092 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5605b450c5e0 0x5605b450b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.175+0000 7f9dbe5be740 4 client.15102 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.175+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3018419092 --> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] -- command(tid 0: {"prefix": "status"}) -- 0x5605b43f24e0 con 0x5605b450c5e0 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.175+0000 7f9db8e42640 1 --2- 172.21.15.83:0/3018419092 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5605b450c5e0 0x5605b450b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9d9cff9640 10 client.15102 ms_handle_connect on v2:172.21.15.111:6836/2927430878 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9db8e42640 1 --2- 172.21.15.83:0/3018419092 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5605b450c5e0 0x5605b450b2f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd1b8c32c2b87581 server_cookie=6a10e1cacc6d9913 in_seq=0 out_seq=0 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9d9cff9640 1 -- 172.21.15.83:0/3018419092 <== mds.2 v2:172.21.15.111:6836/2927430878 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5605b43f24e0 con 0x5605b450c5e0 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9d9cff9640 10 client.15102 handle_command_reply: tid=0 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbe5be740 1 client.15102 shutdown 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3018419092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b4489f90 msgr2=0x5605b4497b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/3018419092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b4489f90 0x5605b4497b40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9db0098630 tx=0x7f9db00cde80 comp rx=0 tx=0).stop 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3018419092 shutdown_connections 2024-04-25T11:43:09.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/3018419092 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5605b450c5e0 0x5605b450b2f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/3018419092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5605b448bb20 0x5605b449ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/3018419092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5605b448b160 0x5605b4498080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9d9cff9640 0 client.15102 ms_handle_reset on v2:172.21.15.111:6836/2927430878 2024-04-25T11:43:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbe5be740 1 --2- 172.21.15.83:0/3018419092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5605b4489f90 0x5605b4497b40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3018419092 >> 172.21.15.83:0/3018419092 conn(0x5605b4481070 msgr2=0x5605b4483fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3018419092 shutdown_connections 2024-04-25T11:43:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbe5be740 1 -- 172.21.15.83:0/3018419092 wait complete. 2024-04-25T11:43:09.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbe5be740 20 client.15102 trim_cache size 0 max 16384 2024-04-25T11:43:09.168 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:43:09.168 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "7fcf54c6-02f7-11ef-bc93-c7b262605968", 2024-04-25T11:43:09.168 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 2, 2024-04-25T11:43:09.168 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24442, 2024-04-25T11:43:09.169 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T11:43:09.169 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T11:43:09.169 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T11:43:09.169 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 80.572672933000007, 2024-04-25T11:43:09.169 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-04-25T11:43:09.169 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-25T11:43:09.169 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T11:43:09.169 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 103.006295202 2024-04-25T11:43:09.169 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:43:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbb8a7640 1 -- 172.21.15.83:0/4011795552 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9da00a3c40 msgr2=0x7f9da00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbb8a7640 1 --2- 172.21.15.83:0/4011795552 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9da00a3c40 0x7f9da00a6100 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9dac02ade0 tx=0x7f9dac036000 comp rx=0 tx=0).stop 2024-04-25T11:43:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbb8a7640 1 -- 172.21.15.83:0/4011795552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4100f30 msgr2=0x7f9db406b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.176+0000 7f9dbb8a7640 1 --2- 172.21.15.83:0/4011795552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4100f30 0x7f9db406b120 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9db000d770 tx=0x7f9db000dc30 comp rx=0 tx=0).stop 2024-04-25T11:43:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.177+0000 7f9dbb8a7640 1 -- 172.21.15.83:0/4011795552 shutdown_connections 2024-04-25T11:43:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.177+0000 7f9dbb8a7640 1 --2- 172.21.15.83:0/4011795552 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9da00a3c40 0x7f9da00a6100 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.177+0000 7f9dbb8a7640 1 --2- 172.21.15.83:0/4011795552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9db4102a40 0x7f9db419f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.177+0000 7f9dbb8a7640 1 --2- 172.21.15.83:0/4011795552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9db4102080 0x7f9db419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.177+0000 7f9dbb8a7640 1 --2- 172.21.15.83:0/4011795552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9db4100f30 0x7f9db406b120 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.177+0000 7f9dbb8a7640 1 -- 172.21.15.83:0/4011795552 >> 172.21.15.83:0/4011795552 conn(0x7f9db40fc4a0 msgr2=0x7f9db4105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.177+0000 7f9dbb8a7640 1 -- 172.21.15.83:0/4011795552 shutdown_connections 2024-04-25T11:43:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.177+0000 7f9dbb8a7640 1 -- 172.21.15.83:0/4011795552 wait complete. 2024-04-25T11:43:09.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T11:43:09.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:09 smithi094 ceph-mon[33032]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:43:09.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:09 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1613964539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:43:09.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:09 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2787721898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:09.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:09 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/536464712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:43:09.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 -- 172.21.15.83:0/3924366233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a78100e90 msgr2=0x7f6a781012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 --2- 172.21.15.83:0/3924366233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a78100e90 0x7f6a781012b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009900 tx=0x7f6a6c02a410 comp rx=0 tx=0).stop 2024-04-25T11:43:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 -- 172.21.15.83:0/3924366233 shutdown_connections 2024-04-25T11:43:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 --2- 172.21.15.83:0/3924366233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a781029a0 0x7f6a781092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 --2- 172.21.15.83:0/3924366233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a78101fe0 0x7f6a78102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 --2- 172.21.15.83:0/3924366233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a78100e90 0x7f6a781012b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 -- 172.21.15.83:0/3924366233 >> 172.21.15.83:0/3924366233 conn(0x7f6a780fc400 msgr2=0x7f6a780fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 -- 172.21.15.83:0/3924366233 shutdown_connections 2024-04-25T11:43:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 -- 172.21.15.83:0/3924366233 wait complete. 2024-04-25T11:43:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 Processor -- start 2024-04-25T11:43:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 -- start start 2024-04-25T11:43:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a78100e90 0x7f6a7810b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a78101fe0 0x7f6a78109e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a781029a0 0x7f6a7810a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7810daf0 con 0x7f6a78100e90 2024-04-25T11:43:09.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7810d970 con 0x7f6a781029a0 2024-04-25T11:43:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7ed87640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7810dc70 con 0x7f6a78101fe0 2024-04-25T11:43:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.286+0000 7f6a7cb23640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a78100e90 0x7f6a7810b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a7cb23640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a78100e90 0x7f6a7810b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32986/0 (socket says 172.21.15.83:32986) 2024-04-25T11:43:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a7cb23640 1 -- 172.21.15.83:0/4189116725 learned_addr learned my addr 172.21.15.83:0/4189116725 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a7d324640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a781029a0 0x7f6a7810a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a77fff640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a78101fe0 0x7f6a78109e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a7cb23640 1 -- 172.21.15.83:0/4189116725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a78101fe0 msgr2=0x7f6a78109e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a7cb23640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a78101fe0 0x7f6a78109e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a7cb23640 1 -- 172.21.15.83:0/4189116725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a781029a0 msgr2=0x7f6a7810a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a7cb23640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a781029a0 0x7f6a7810a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a7cb23640 1 -- 172.21.15.83:0/4189116725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7810ac40 con 0x7f6a78100e90 2024-04-25T11:43:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a77fff640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a78101fe0 0x7f6a78109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a7d324640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a781029a0 0x7f6a7810a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a7cb23640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a78100e90 0x7f6a7810b830 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c005940 tx=0x7f6a6c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a75ffb640 1 -- 172.21.15.83:0/4189116725 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c03f030 con 0x7f6a78100e90 2024-04-25T11:43:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a75ffb640 1 -- 172.21.15.83:0/4189116725 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a6c003cb0 con 0x7f6a78100e90 2024-04-25T11:43:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a75ffb640 1 -- 172.21.15.83:0/4189116725 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c035070 con 0x7f6a78100e90 2024-04-25T11:43:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a7ed87640 1 -- 172.21.15.83:0/4189116725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a781a2230 con 0x7f6a78100e90 2024-04-25T11:43:09.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.287+0000 7f6a7ed87640 1 -- 172.21.15.83:0/4189116725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a781a2780 con 0x7f6a78100e90 2024-04-25T11:43:09.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.289+0000 7f6a75ffb640 1 -- 172.21.15.83:0/4189116725 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6a6c042030 con 0x7f6a78100e90 2024-04-25T11:43:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a81b7a740 1 Processor -- start 2024-04-25T11:43:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a81b7a740 1 -- start start 2024-04-25T11:43:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a81b7a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 0x55a9d571efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a81b7a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 0x55a9d57201d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a81b7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5726fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a81b7a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a9d5683c50 con 0x55a9d5720710 2024-04-25T11:43:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a81b7a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a9d5620830 con 0x55a9d571fd50 2024-04-25T11:43:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a81b7a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a9d567fc10 con 0x55a9d571eb80 2024-04-25T11:43:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a75ffb640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f6a680a3c40 0x7f6a680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a77fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5726fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a77fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5726fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33000/0 (socket says 172.21.15.83:33000) 2024-04-25T11:43:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a77fff640 1 -- 172.21.15.83:0/1561405323 learned_addr learned my addr 172.21.15.83:0/1561405323 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a7cb23640 1 --2- 172.21.15.83:0/1561405323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 0x55a9d57201d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a7d324640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f6a680a3c40 0x7f6a680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a7d324640 1 --2- 172.21.15.83:0/1561405323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 0x55a9d571efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a75ffb640 1 -- 172.21.15.83:0/4189116725 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f6a6c0f0f80 con 0x7f6a78100e90 2024-04-25T11:43:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a77fff640 1 -- 172.21.15.83:0/1561405323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 msgr2=0x55a9d571efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a77fff640 1 --2- 172.21.15.83:0/1561405323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 0x55a9d571efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a77fff640 1 -- 172.21.15.83:0/1561405323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 msgr2=0x55a9d57201d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a77fff640 1 --2- 172.21.15.83:0/1561405323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 0x55a9d57201d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a77fff640 1 -- 172.21.15.83:0/1561405323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9d53962d0 con 0x55a9d5720710 2024-04-25T11:43:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.290+0000 7f6a7d324640 1 --2- 172.21.15.83:0/1561405323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 0x55a9d571efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.291+0000 7f6a77fff640 1 --2- 172.21.15.83:0/1561405323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5726fa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400bbc0 tx=0x7f6a6400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a32eb72e185b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.291+0000 7f6a7d324640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f6a680a3c40 0x7f6a680a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55a9d571fc50 tx=0x7f6a700095b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.291+0000 7f6a62ffd640 1 -- 172.21.15.83:0/1561405323 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a64004220 con 0x55a9d5720710 2024-04-25T11:43:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.291+0000 7f6a62ffd640 1 -- 172.21.15.83:0/1561405323 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a640043c0 con 0x55a9d5720710 2024-04-25T11:43:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.291+0000 7f6a62ffd640 1 -- 172.21.15.83:0/1561405323 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6400f8a0 con 0x55a9d5720710 2024-04-25T11:43:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.291+0000 7f6a81b7a740 1 -- 172.21.15.83:0/1561405323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 msgr2=0x55a9d5726fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.291+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/1561405323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5726fa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400bbc0 tx=0x7f6a6400d710 comp rx=0 tx=0).stop 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.291+0000 7f6a81b7a740 1 -- 172.21.15.83:0/1561405323 shutdown_connections 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.291+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/1561405323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5726fa0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.291+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/1561405323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 0x55a9d57201d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.291+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/1561405323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 0x55a9d571efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.291+0000 7f6a81b7a740 1 -- 172.21.15.83:0/1561405323 >> 172.21.15.83:0/1561405323 conn(0x55a9d571a070 msgr2=0x55a9d571a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.291+0000 7f6a81b7a740 1 -- 172.21.15.83:0/1561405323 shutdown_connections 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.291+0000 7f6a81b7a740 1 -- 172.21.15.83:0/1561405323 wait complete. 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.292+0000 7f6a81b7a740 1 -- start start 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.292+0000 7f6a81b7a740 1 Processor -- start 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.292+0000 7f6a81b7a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571eb80 0x55a9d5728fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.292+0000 7f6a81b7a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571fd50 0x55a9d57294e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.292+0000 7f6a81b7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5730520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.292+0000 7f6a81b7a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a9d5727720 con 0x55a9d5720710 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.292+0000 7f6a81b7a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a9d561e470 con 0x55a9d571eb80 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.292+0000 7f6a81b7a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a9d57278a0 con 0x55a9d571fd50 2024-04-25T11:43:09.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.292+0000 7f6a81b7a740 10 client.0 _ll_register_callbacks cb 0x55a9d5643f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:09.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.292+0000 7f6a7d324640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5730520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.292+0000 7f6a7d324640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5730520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33012/0 (socket says 172.21.15.83:33012) 2024-04-25T11:43:09.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.292+0000 7f6a7d324640 1 -- 172.21.15.83:0/3546020093 learned_addr learned my addr 172.21.15.83:0/3546020093 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.292+0000 7f6a7cb23640 1 --2- 172.21.15.83:0/3546020093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571eb80 0x55a9d5728fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a7d324640 1 -- 172.21.15.83:0/3546020093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571fd50 msgr2=0x55a9d57294e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a7d324640 1 --2- 172.21.15.83:0/3546020093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571fd50 0x55a9d57294e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a7d324640 1 -- 172.21.15.83:0/3546020093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571eb80 msgr2=0x55a9d5728fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a7d324640 1 --2- 172.21.15.83:0/3546020093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571eb80 0x55a9d5728fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a7d324640 1 -- 172.21.15.83:0/3546020093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9d54606b0 con 0x55a9d5720710 2024-04-25T11:43:09.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a7cb23640 1 --2- 172.21.15.83:0/3546020093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571eb80 0x55a9d5728fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:09.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a7d324640 1 --2- 172.21.15.83:0/3546020093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5730520 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6a70010ae0 tx=0x7f6a70004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53aa9ae07c95d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a57fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:09.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a57fff640 1 -- 172.21.15.83:0/3546020093 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7002d060 con 0x55a9d5720710 2024-04-25T11:43:09.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a57fff640 1 -- 172.21.15.83:0/3546020093 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f6a7001ea40 con 0x55a9d5720710 2024-04-25T11:43:09.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a81b7a740 1 -- 172.21.15.83:0/3546020093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a640043c0 con 0x55a9d5720710 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a57fff640 1 -- 172.21.15.83:0/3546020093 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6a7001ec40 con 0x55a9d5720710 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a81b7a740 10 client.15114 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a81b7a740 1 -- 172.21.15.83:0/3546020093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a9d5733960 con 0x55a9d5720710 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a57fff640 1 -- 172.21.15.83:0/3546020093 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a640043c0 con 0x55a9d5720710 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a57fff640 1 -- 172.21.15.83:0/3546020093 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f6a700113e0 con 0x55a9d5720710 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a81b7a740 10 client.15114 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a81b7a740 10 client.15114 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a81b7a740 20 client.15114 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/3546020093 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55a9d5736090 0x55a9d5737500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a81b7a740 4 client.15114 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.293+0000 7f6a81b7a740 1 -- 172.21.15.83:0/3546020093 --> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9d5691670 con 0x55a9d5736090 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.294+0000 7f6a7cb23640 1 --2- 172.21.15.83:0/3546020093 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55a9d5736090 0x55a9d5737500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.294+0000 7f6a57fff640 10 client.15114 ms_handle_connect on v2:172.21.15.111:6838/2233531151 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.294+0000 7f6a7cb23640 1 --2- 172.21.15.83:0/3546020093 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55a9d5736090 0x55a9d5737500 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b722c40e3a62e0bc server_cookie=bddcca3add5175ff in_seq=0 out_seq=0 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.295+0000 7f6a57fff640 1 -- 172.21.15.83:0/3546020093 <== mds.1 v2:172.21.15.111:6838/2233531151 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9d5691670 con 0x55a9d5736090 2024-04-25T11:43:09.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.295+0000 7f6a57fff640 10 client.15114 handle_command_reply: tid=0 2024-04-25T11:43:09.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.295+0000 7f6a81b7a740 1 client.15114 shutdown 2024-04-25T11:43:09.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.295+0000 7f6a81b7a740 1 -- 172.21.15.83:0/3546020093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 msgr2=0x55a9d5730520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.295+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/3546020093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5730520 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6a70010ae0 tx=0x7f6a70004ab0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.295+0000 7f6a81b7a740 1 -- 172.21.15.83:0/3546020093 shutdown_connections 2024-04-25T11:43:09.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.295+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/3546020093 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55a9d5736090 0x55a9d5737500 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.295+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/3546020093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5730520 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.295+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/3546020093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571fd50 0x55a9d57294e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.295+0000 7f6a57fff640 0 client.15114 ms_handle_reset on v2:172.21.15.111:6838/2233531151 2024-04-25T11:43:09.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.295+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/3546020093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571eb80 0x55a9d5728fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.295+0000 7f6a81b7a740 1 -- 172.21.15.83:0/3546020093 >> 172.21.15.83:0/3546020093 conn(0x55a9d571a070 msgr2=0x55a9d57191d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.295+0000 7f6a81b7a740 1 -- 172.21.15.83:0/3546020093 shutdown_connections 2024-04-25T11:43:09.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.295+0000 7f6a81b7a740 1 -- 172.21.15.83:0/3546020093 wait complete. 2024-04-25T11:43:09.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.295+0000 7f6a81b7a740 20 client.15114 trim_cache size 0 max 16384 2024-04-25T11:43:09.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a81b7a740 1 Processor -- start 2024-04-25T11:43:09.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a81b7a740 1 -- start start 2024-04-25T11:43:09.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a81b7a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 0x55a9d5720130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a81b7a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 0x55a9d57304e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a81b7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5719050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a81b7a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a9d5735840 con 0x55a9d5720710 2024-04-25T11:43:09.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a81b7a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a9d5735f10 con 0x55a9d571fd50 2024-04-25T11:43:09.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a81b7a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a9d57365e0 con 0x55a9d571eb80 2024-04-25T11:43:09.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a7d324640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5719050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a7d324640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5719050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33022/0 (socket says 172.21.15.83:33022) 2024-04-25T11:43:09.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a7d324640 1 -- 172.21.15.83:0/2485393780 learned_addr learned my addr 172.21.15.83:0/2485393780 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a7d324640 1 -- 172.21.15.83:0/2485393780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 msgr2=0x55a9d5720130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a7d324640 1 --2- 172.21.15.83:0/2485393780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 0x55a9d5720130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a7d324640 1 -- 172.21.15.83:0/2485393780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 msgr2=0x55a9d57304e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a77fff640 1 --2- 172.21.15.83:0/2485393780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 0x55a9d57304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a7d324640 1 --2- 172.21.15.83:0/2485393780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 0x55a9d57304e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a7d324640 1 -- 172.21.15.83:0/2485393780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9d571f7d0 con 0x55a9d5720710 2024-04-25T11:43:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a77fff640 1 --2- 172.21.15.83:0/2485393780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 0x55a9d57304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.313+0000 7f6a7d324640 1 --2- 172.21.15.83:0/2485393780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5719050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009d20 tx=0x7f6a700107f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd0c2fafe218afc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.314+0000 7f6a627fc640 1 -- 172.21.15.83:0/2485393780 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70037070 con 0x55a9d5720710 2024-04-25T11:43:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.314+0000 7f6a627fc640 1 -- 172.21.15.83:0/2485393780 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a70011ac0 con 0x55a9d5720710 2024-04-25T11:43:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.314+0000 7f6a627fc640 1 -- 172.21.15.83:0/2485393780 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70011570 con 0x55a9d5720710 2024-04-25T11:43:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.314+0000 7f6a81b7a740 1 -- 172.21.15.83:0/2485393780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 msgr2=0x55a9d5719050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.314+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/2485393780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5719050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009d20 tx=0x7f6a700107f0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.314+0000 7f6a81b7a740 1 -- 172.21.15.83:0/2485393780 shutdown_connections 2024-04-25T11:43:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.314+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/2485393780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d5719050 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.314+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/2485393780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 0x55a9d57304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.314+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/2485393780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 0x55a9d5720130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.314+0000 7f6a81b7a740 1 -- 172.21.15.83:0/2485393780 >> 172.21.15.83:0/2485393780 conn(0x55a9d571a070 msgr2=0x55a9d5718940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.314+0000 7f6a81b7a740 1 -- 172.21.15.83:0/2485393780 shutdown_connections 2024-04-25T11:43:09.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.314+0000 7f6a81b7a740 1 -- 172.21.15.83:0/2485393780 wait complete. 2024-04-25T11:43:09.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.314+0000 7f6a81b7a740 1 -- start start 2024-04-25T11:43:09.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a81b7a740 1 Processor -- start 2024-04-25T11:43:09.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a81b7a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 0x55a9d572c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a81b7a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 0x55a9d572cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a81b7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d572fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a81b7a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a9d5737380 con 0x55a9d5720710 2024-04-25T11:43:09.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a81b7a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55a9d5736cb0 con 0x55a9d571fd50 2024-04-25T11:43:09.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a81b7a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a9d5737a50 con 0x55a9d571eb80 2024-04-25T11:43:09.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a81b7a740 10 client.0 _ll_register_callbacks cb 0x7f6a70009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:09.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a7d324640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d572fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a7d324640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d572fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33038/0 (socket says 172.21.15.83:33038) 2024-04-25T11:43:09.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a7d324640 1 -- 172.21.15.83:0/874500198 learned_addr learned my addr 172.21.15.83:0/874500198 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a77fff640 1 --2- 172.21.15.83:0/874500198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 0x55a9d572cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a7cb23640 1 --2- 172.21.15.83:0/874500198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 0x55a9d572c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a7d324640 1 -- 172.21.15.83:0/874500198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 msgr2=0x55a9d572c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a7d324640 1 --2- 172.21.15.83:0/874500198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 0x55a9d572c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a7d324640 1 -- 172.21.15.83:0/874500198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 msgr2=0x55a9d572cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a7d324640 1 --2- 172.21.15.83:0/874500198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 0x55a9d572cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a7d324640 1 -- 172.21.15.83:0/874500198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9d5730190 con 0x55a9d5720710 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a7cb23640 1 --2- 172.21.15.83:0/874500198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 0x55a9d572c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a77fff640 1 --2- 172.21.15.83:0/874500198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 0x55a9d572cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a7d324640 1 --2- 172.21.15.83:0/874500198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d572fb20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009e10 tx=0x7f6a70011520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbe5b5037c30318 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.315+0000 7f6a57fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.316+0000 7f6a57fff640 1 -- 172.21.15.83:0/874500198 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7003f030 con 0x55a9d5720710 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.316+0000 7f6a57fff640 1 -- 172.21.15.83:0/874500198 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f6a7001ea40 con 0x55a9d5720710 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.316+0000 7f6a81b7a740 1 -- 172.21.15.83:0/874500198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a70011ac0 con 0x55a9d5720710 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.316+0000 7f6a57fff640 1 -- 172.21.15.83:0/874500198 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f6a70037070 con 0x55a9d5720710 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.316+0000 7f6a81b7a740 10 client.15120 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.316+0000 7f6a81b7a740 1 -- 172.21.15.83:0/874500198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a9d579ca30 con 0x55a9d5720710 2024-04-25T11:43:09.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.316+0000 7f6a57fff640 1 -- 172.21.15.83:0/874500198 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a70011ac0 con 0x55a9d5720710 2024-04-25T11:43:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.316+0000 7f6a57fff640 1 -- 172.21.15.83:0/874500198 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f6a70005850 con 0x55a9d5720710 2024-04-25T11:43:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.316+0000 7f6a81b7a740 10 client.15120 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.316+0000 7f6a81b7a740 10 client.15120 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T11:43:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.316+0000 7f6a81b7a740 20 client.15120 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.316+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/874500198 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55a9d57a1310 0x55a9d57a0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.316+0000 7f6a81b7a740 4 client.15120 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:43:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.316+0000 7f6a81b7a740 1 -- 172.21.15.83:0/874500198 --> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] -- command(tid 0: {"prefix": "status"}) -- 0x55a9d568b4e0 con 0x55a9d57a1310 2024-04-25T11:43:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.316+0000 7f6a7cb23640 1 --2- 172.21.15.83:0/874500198 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55a9d57a1310 0x55a9d57a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a57fff640 10 client.15120 ms_handle_connect on v2:172.21.15.111:6838/2233531151 2024-04-25T11:43:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a7cb23640 1 --2- 172.21.15.83:0/874500198 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55a9d57a1310 0x55a9d57a0020 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fbceffb736b4a5f9 server_cookie=8a88cd1e8c843ff6 in_seq=0 out_seq=0 2024-04-25T11:43:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a57fff640 1 -- 172.21.15.83:0/874500198 <== mds.1 v2:172.21.15.111:6838/2233531151 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55a9d568b4e0 con 0x55a9d57a1310 2024-04-25T11:43:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a57fff640 10 client.15120 handle_command_reply: tid=0 2024-04-25T11:43:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a81b7a740 1 client.15120 shutdown 2024-04-25T11:43:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a81b7a740 1 -- 172.21.15.83:0/874500198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 msgr2=0x55a9d572fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/874500198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d572fb20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009e10 tx=0x7f6a70011520 comp rx=0 tx=0).stop 2024-04-25T11:43:09.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a81b7a740 1 -- 172.21.15.83:0/874500198 shutdown_connections 2024-04-25T11:43:09.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/874500198 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55a9d57a1310 0x55a9d57a0020 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/874500198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a9d5720710 0x55a9d572fb20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/874500198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a9d571fd50 0x55a9d572cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a57fff640 0 client.15120 ms_handle_reset on v2:172.21.15.111:6838/2233531151 2024-04-25T11:43:09.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a81b7a740 1 --2- 172.21.15.83:0/874500198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a9d571eb80 0x55a9d572c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a81b7a740 1 -- 172.21.15.83:0/874500198 >> 172.21.15.83:0/874500198 conn(0x55a9d571a070 msgr2=0x55a9d57191d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a81b7a740 1 -- 172.21.15.83:0/874500198 shutdown_connections 2024-04-25T11:43:09.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a81b7a740 1 -- 172.21.15.83:0/874500198 wait complete. 2024-04-25T11:43:09.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.317+0000 7f6a81b7a740 20 client.15120 trim_cache size 0 max 16384 2024-04-25T11:43:09.310 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:43:09.311 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "7fcf54c6-02f7-11ef-bc93-c7b262605968", 2024-04-25T11:43:09.311 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 1, 2024-04-25T11:43:09.311 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24469, 2024-04-25T11:43:09.311 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T11:43:09.311 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T11:43:09.311 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T11:43:09.311 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 81.728056252000002, 2024-04-25T11:43:09.311 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-04-25T11:43:09.311 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-25T11:43:09.311 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T11:43:09.311 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 97.601938124 2024-04-25T11:43:09.311 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:43:09.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.318+0000 7f6a7ed87640 1 -- 172.21.15.83:0/4189116725 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f6a680a3c40 msgr2=0x7f6a680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.318+0000 7f6a7ed87640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f6a680a3c40 0x7f6a680a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55a9d571fc50 tx=0x7f6a700095b0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.318+0000 7f6a7ed87640 1 -- 172.21.15.83:0/4189116725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a78100e90 msgr2=0x7f6a7810b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.318+0000 7f6a7ed87640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a78100e90 0x7f6a7810b830 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c005940 tx=0x7f6a6c004430 comp rx=0 tx=0).stop 2024-04-25T11:43:09.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.318+0000 7f6a7ed87640 1 -- 172.21.15.83:0/4189116725 shutdown_connections 2024-04-25T11:43:09.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.318+0000 7f6a7ed87640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f6a680a3c40 0x7f6a680a6100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.318+0000 7f6a7ed87640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a781029a0 0x7f6a7810a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.318+0000 7f6a7ed87640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a78101fe0 0x7f6a78109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.318+0000 7f6a7ed87640 1 --2- 172.21.15.83:0/4189116725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a78100e90 0x7f6a7810b830 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.318+0000 7f6a7ed87640 1 -- 172.21.15.83:0/4189116725 >> 172.21.15.83:0/4189116725 conn(0x7f6a780fc400 msgr2=0x7f6a78105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.318+0000 7f6a7ed87640 1 -- 172.21.15.83:0/4189116725 shutdown_connections 2024-04-25T11:43:09.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.318+0000 7f6a7ed87640 1 -- 172.21.15.83:0/4189116725 wait complete. 2024-04-25T11:43:09.315 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T11:43:09.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:09 smithi111 ceph-mon[32784]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:43:09.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:09 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1613964539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:43:09.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:09 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2787721898' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:09.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:09 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/536464712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:43:09.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.426+0000 7f8d038d0640 1 -- 172.21.15.83:0/1721786007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc102a30 msgr2=0x7f8cfc104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.426+0000 7f8d038d0640 1 --2- 172.21.15.83:0/1721786007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc102a30 0x7f8cfc104e20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009900 tx=0x7f8cf802a410 comp rx=0 tx=0).stop 2024-04-25T11:43:09.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.426+0000 7f8d038d0640 1 -- 172.21.15.83:0/1721786007 shutdown_connections 2024-04-25T11:43:09.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.426+0000 7f8d038d0640 1 --2- 172.21.15.83:0/1721786007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc107c90 0x7f8cfc10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.426+0000 7f8d038d0640 1 --2- 172.21.15.83:0/1721786007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cfc105360 0x7f8cfc107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.426+0000 7f8d038d0640 1 --2- 172.21.15.83:0/1721786007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc102a30 0x7f8cfc104e20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.426+0000 7f8d038d0640 1 -- 172.21.15.83:0/1721786007 >> 172.21.15.83:0/1721786007 conn(0x7f8cfc0fc440 msgr2=0x7f8cfc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.426+0000 7f8d038d0640 1 -- 172.21.15.83:0/1721786007 shutdown_connections 2024-04-25T11:43:09.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.426+0000 7f8d038d0640 1 -- 172.21.15.83:0/1721786007 wait complete. 2024-04-25T11:43:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d038d0640 1 Processor -- start 2024-04-25T11:43:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d038d0640 1 -- start start 2024-04-25T11:43:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d038d0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc102a30 0x7f8cfc102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d038d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cfc105360 0x7f8cfc1009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d038d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc107c90 0x7f8cfc100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d038d0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc10daf0 con 0x7f8cfc107c90 2024-04-25T11:43:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d038d0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc10d970 con 0x7f8cfc105360 2024-04-25T11:43:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d038d0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8cfc10dc70 con 0x7f8cfc102a30 2024-04-25T11:43:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d01e6d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc107c90 0x7f8cfc100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d01e6d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc107c90 0x7f8cfc100f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33050/0 (socket says 172.21.15.83:33050) 2024-04-25T11:43:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d01e6d640 1 -- 172.21.15.83:0/603222754 learned_addr learned my addr 172.21.15.83:0/603222754 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d00e6b640 1 --2- 172.21.15.83:0/603222754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cfc105360 0x7f8cfc1009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d01e6d640 1 -- 172.21.15.83:0/603222754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc102a30 msgr2=0x7f8cfc102350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d0166c640 1 --2- 172.21.15.83:0/603222754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc102a30 0x7f8cfc102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d01e6d640 1 --2- 172.21.15.83:0/603222754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc102a30 0x7f8cfc102350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d01e6d640 1 -- 172.21.15.83:0/603222754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cfc105360 msgr2=0x7f8cfc1009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d01e6d640 1 --2- 172.21.15.83:0/603222754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cfc105360 0x7f8cfc1009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.427+0000 7f8d01e6d640 1 -- 172.21.15.83:0/603222754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cfc1017e0 con 0x7f8cfc107c90 2024-04-25T11:43:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.428+0000 7f8d01e6d640 1 --2- 172.21.15.83:0/603222754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc107c90 0x7f8cfc100f20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400cad0 tx=0x7f8cf400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.428+0000 7f8cf27fc640 1 -- 172.21.15.83:0/603222754 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf400ae20 con 0x7f8cfc107c90 2024-04-25T11:43:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.428+0000 7f8cf27fc640 1 -- 172.21.15.83:0/603222754 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cf4005020 con 0x7f8cfc107c90 2024-04-25T11:43:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.428+0000 7f8d038d0640 1 -- 172.21.15.83:0/603222754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cfc1a2380 con 0x7f8cfc107c90 2024-04-25T11:43:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.428+0000 7f8cf27fc640 1 -- 172.21.15.83:0/603222754 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf400f6b0 con 0x7f8cfc107c90 2024-04-25T11:43:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.428+0000 7f8d038d0640 1 -- 172.21.15.83:0/603222754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cfc1a28f0 con 0x7f8cfc107c90 2024-04-25T11:43:09.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.430+0000 7f8cf27fc640 1 -- 172.21.15.83:0/603222754 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8cf40051d0 con 0x7f8cfc107c90 2024-04-25T11:43:09.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.430+0000 7f8d065e2740 1 Processor -- start 2024-04-25T11:43:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.430+0000 7f8d065e2740 1 -- start start 2024-04-25T11:43:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.430+0000 7f8cf27fc640 1 --2- 172.21.15.83:0/603222754 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8ce80a3cc0 0x7f8ce80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.430+0000 7f8d065e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d7b80 0x556a326d7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d065e2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d8d50 0x556a326d91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d065e2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d9710 0x556a326dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d065e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556a3263cc50 con 0x556a326d7b80 2024-04-25T11:43:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d065e2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556a325d9830 con 0x556a326d8d50 2024-04-25T11:43:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d065e2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556a32638c10 con 0x556a326d9710 2024-04-25T11:43:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d0166c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d7b80 0x556a326d7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8cf27fc640 1 -- 172.21.15.83:0/603222754 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f8cf40d0490 con 0x7f8cfc107c90 2024-04-25T11:43:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d0166c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d7b80 0x556a326d7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33064/0 (socket says 172.21.15.83:33064) 2024-04-25T11:43:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d0166c640 1 -- 172.21.15.83:0/2919451711 learned_addr learned my addr 172.21.15.83:0/2919451711 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d00e6b640 1 --2- 172.21.15.83:0/603222754 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8ce80a3cc0 0x7f8ce80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d01e6d640 1 --2- 172.21.15.83:0/2919451711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d9710 0x556a326dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d00e6b640 1 --2- 172.21.15.83:0/2919451711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d8d50 0x556a326d91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d0166c640 1 -- 172.21.15.83:0/2919451711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d9710 msgr2=0x556a326dffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d0166c640 1 --2- 172.21.15.83:0/2919451711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d9710 0x556a326dffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d0166c640 1 -- 172.21.15.83:0/2919451711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d8d50 msgr2=0x556a326d91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d0166c640 1 --2- 172.21.15.83:0/2919451711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d8d50 0x556a326d91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d0166c640 1 -- 172.21.15.83:0/2919451711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a3234f2d0 con 0x556a326d7b80 2024-04-25T11:43:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d00e6b640 1 --2- 172.21.15.83:0/2919451711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d8d50 0x556a326d91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d0166c640 1 --2- 172.21.15.83:0/2919451711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d7b80 0x556a326d7fa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8006010 tx=0x7f8cf8032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a79d8e1c1361d34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8d00e6b640 1 --2- 172.21.15.83:0/603222754 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8ce80a3cc0 0x7f8ce80a6180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc1020c0 tx=0x7f8cec00b930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8ce77fe640 1 -- 172.21.15.83:0/2919451711 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf803f030 con 0x556a326d7b80 2024-04-25T11:43:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8ce77fe640 1 -- 172.21.15.83:0/2919451711 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cf8032dc0 con 0x556a326d7b80 2024-04-25T11:43:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.431+0000 7f8ce77fe640 1 -- 172.21.15.83:0/2919451711 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8035070 con 0x556a326d7b80 2024-04-25T11:43:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.432+0000 7f8d065e2740 1 -- 172.21.15.83:0/2919451711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d7b80 msgr2=0x556a326d7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.432+0000 7f8d065e2740 1 --2- 172.21.15.83:0/2919451711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d7b80 0x556a326d7fa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8006010 tx=0x7f8cf8032ba0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.432+0000 7f8d065e2740 1 -- 172.21.15.83:0/2919451711 shutdown_connections 2024-04-25T11:43:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.432+0000 7f8d065e2740 1 --2- 172.21.15.83:0/2919451711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d9710 0x556a326dffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.432+0000 7f8d065e2740 1 --2- 172.21.15.83:0/2919451711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d8d50 0x556a326d91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.432+0000 7f8d065e2740 1 --2- 172.21.15.83:0/2919451711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d7b80 0x556a326d7fa0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.432+0000 7f8d065e2740 1 -- 172.21.15.83:0/2919451711 >> 172.21.15.83:0/2919451711 conn(0x556a326d3070 msgr2=0x556a326d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.432+0000 7f8d065e2740 1 -- 172.21.15.83:0/2919451711 shutdown_connections 2024-04-25T11:43:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.432+0000 7f8d065e2740 1 -- 172.21.15.83:0/2919451711 wait complete. 2024-04-25T11:43:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.432+0000 7f8d065e2740 1 -- start start 2024-04-25T11:43:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.432+0000 7f8d065e2740 1 Processor -- start 2024-04-25T11:43:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d065e2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d7b80 0x556a326e1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d065e2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d8d50 0x556a326e2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d065e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d9710 0x556a326e9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d065e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556a326e06c0 con 0x556a326d9710 2024-04-25T11:43:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d065e2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556a325d7470 con 0x556a326d8d50 2024-04-25T11:43:09.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d065e2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556a326e0840 con 0x556a326d7b80 2024-04-25T11:43:09.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d065e2740 10 client.0 _ll_register_callbacks cb 0x556a325fcf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:09.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d01e6d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d9710 0x556a326e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d01e6d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d9710 0x556a326e9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33072/0 (socket says 172.21.15.83:33072) 2024-04-25T11:43:09.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d01e6d640 1 -- 172.21.15.83:0/835191180 learned_addr learned my addr 172.21.15.83:0/835191180 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d0166c640 1 --2- 172.21.15.83:0/835191180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d7b80 0x556a326e1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d01e6d640 1 -- 172.21.15.83:0/835191180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d7b80 msgr2=0x556a326e1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d01e6d640 1 --2- 172.21.15.83:0/835191180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d7b80 0x556a326e1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d01e6d640 1 -- 172.21.15.83:0/835191180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d8d50 msgr2=0x556a326e2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d01e6d640 1 --2- 172.21.15.83:0/835191180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d8d50 0x556a326e2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d01e6d640 1 -- 172.21.15.83:0/835191180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a324196b0 con 0x556a326d9710 2024-04-25T11:43:09.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d01e6d640 1 --2- 172.21.15.83:0/835191180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d9710 0x556a326e9490 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40cde80 tx=0x7f8cf409cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe60cbdd1084eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8ce4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:09.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8ce4ff9640 1 -- 172.21.15.83:0/835191180 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf40a09d0 con 0x556a326d9710 2024-04-25T11:43:09.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8ce4ff9640 1 -- 172.21.15.83:0/835191180 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f8cf40054f0 con 0x556a326d9710 2024-04-25T11:43:09.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.433+0000 7f8d065e2740 1 -- 172.21.15.83:0/835191180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cf8032dc0 con 0x556a326d9710 2024-04-25T11:43:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.434+0000 7f8ce4ff9640 1 -- 172.21.15.83:0/835191180 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8cf4053020 con 0x556a326d9710 2024-04-25T11:43:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.434+0000 7f8d065e2740 10 client.15132 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.434+0000 7f8d065e2740 1 -- 172.21.15.83:0/835191180 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556a326ec7e0 con 0x556a326d9710 2024-04-25T11:43:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.434+0000 7f8ce4ff9640 1 -- 172.21.15.83:0/835191180 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cf8032dc0 con 0x556a326d9710 2024-04-25T11:43:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.434+0000 7f8ce4ff9640 1 -- 172.21.15.83:0/835191180 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f8cf40a4e00 con 0x556a326d9710 2024-04-25T11:43:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.434+0000 7f8d065e2740 10 client.15132 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.434+0000 7f8d065e2740 10 client.15132 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:43:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.434+0000 7f8d065e2740 20 client.15132 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.434+0000 7f8d065e2740 1 --2- 172.21.15.83:0/835191180 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x556a326eee70 0x556a326f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.434+0000 7f8d065e2740 4 client.15132 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:43:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.434+0000 7f8d065e2740 1 -- 172.21.15.83:0/835191180 --> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a3264a670 con 0x556a326eee70 2024-04-25T11:43:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.434+0000 7f8d0166c640 1 --2- 172.21.15.83:0/835191180 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x556a326eee70 0x556a326f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.435+0000 7f8ce4ff9640 10 client.15132 ms_handle_connect on v2:172.21.15.94:6838/1461078471 2024-04-25T11:43:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.435+0000 7f8d0166c640 1 --2- 172.21.15.83:0/835191180 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x556a326eee70 0x556a326f02e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e501f1ae3273a0 server_cookie=26ee6fb844181e6b in_seq=0 out_seq=0 2024-04-25T11:43:09.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.435+0000 7f8ce4ff9640 1 -- 172.21.15.83:0/835191180 <== mds.0 v2:172.21.15.94:6838/1461078471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a3264a670 con 0x556a326eee70 2024-04-25T11:43:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.435+0000 7f8ce4ff9640 10 client.15132 handle_command_reply: tid=0 2024-04-25T11:43:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.435+0000 7f8d065e2740 1 client.15132 shutdown 2024-04-25T11:43:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.436+0000 7f8d065e2740 1 -- 172.21.15.83:0/835191180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d9710 msgr2=0x556a326e9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.436+0000 7f8d065e2740 1 --2- 172.21.15.83:0/835191180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d9710 0x556a326e9490 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40cde80 tx=0x7f8cf409cd20 comp rx=0 tx=0).stop 2024-04-25T11:43:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.436+0000 7f8d065e2740 1 -- 172.21.15.83:0/835191180 shutdown_connections 2024-04-25T11:43:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.436+0000 7f8d065e2740 1 --2- 172.21.15.83:0/835191180 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x556a326eee70 0x556a326f02e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.436+0000 7f8d065e2740 1 --2- 172.21.15.83:0/835191180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d9710 0x556a326e9490 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.436+0000 7f8d065e2740 1 --2- 172.21.15.83:0/835191180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d8d50 0x556a326e2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.436+0000 7f8ce4ff9640 0 client.15132 ms_handle_reset on v2:172.21.15.94:6838/1461078471 2024-04-25T11:43:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.436+0000 7f8d065e2740 1 --2- 172.21.15.83:0/835191180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d7b80 0x556a326e1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.436+0000 7f8d065e2740 1 -- 172.21.15.83:0/835191180 >> 172.21.15.83:0/835191180 conn(0x556a326d3070 msgr2=0x556a326d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.436+0000 7f8d065e2740 1 -- 172.21.15.83:0/835191180 shutdown_connections 2024-04-25T11:43:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.436+0000 7f8d065e2740 1 -- 172.21.15.83:0/835191180 wait complete. 2024-04-25T11:43:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.436+0000 7f8d065e2740 20 client.15132 trim_cache size 0 max 16384 2024-04-25T11:43:09.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d065e2740 1 Processor -- start 2024-04-25T11:43:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d065e2740 1 -- start start 2024-04-25T11:43:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d065e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d7b80 0x556a326edc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d065e2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d8d50 0x556a326e9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d065e2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d9710 0x556a326e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d065e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556a326ee620 con 0x556a326d7b80 2024-04-25T11:43:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d065e2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556a326eecf0 con 0x556a326d9710 2024-04-25T11:43:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d065e2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556a326ef3c0 con 0x556a326d8d50 2024-04-25T11:43:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d0166c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d7b80 0x556a326edc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d0166c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d7b80 0x556a326edc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33076/0 (socket says 172.21.15.83:33076) 2024-04-25T11:43:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d0166c640 1 -- 172.21.15.83:0/1890480074 learned_addr learned my addr 172.21.15.83:0/1890480074 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d0166c640 1 -- 172.21.15.83:0/1890480074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d8d50 msgr2=0x556a326e9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d0166c640 1 --2- 172.21.15.83:0/1890480074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d8d50 0x556a326e9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d0166c640 1 -- 172.21.15.83:0/1890480074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d9710 msgr2=0x556a326e1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d0166c640 1 --2- 172.21.15.83:0/1890480074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d9710 0x556a326e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d0166c640 1 -- 172.21.15.83:0/1890480074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a326d87d0 con 0x556a326d7b80 2024-04-25T11:43:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8d0166c640 1 --2- 172.21.15.83:0/1890480074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d7b80 0x556a326edc50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x556a326d8c50 tx=0x7f8cf802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cdc58b2594f6fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8ce6ffd640 1 -- 172.21.15.83:0/1890480074 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8051070 con 0x556a326d7b80 2024-04-25T11:43:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8ce6ffd640 1 -- 172.21.15.83:0/1890480074 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cf804cdc0 con 0x556a326d7b80 2024-04-25T11:43:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.454+0000 7f8ce6ffd640 1 -- 172.21.15.83:0/1890480074 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8035040 con 0x556a326d7b80 2024-04-25T11:43:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.455+0000 7f8d065e2740 1 -- 172.21.15.83:0/1890480074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d7b80 msgr2=0x556a326edc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.455+0000 7f8d065e2740 1 --2- 172.21.15.83:0/1890480074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d7b80 0x556a326edc50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x556a326d8c50 tx=0x7f8cf802a410 comp rx=0 tx=0).stop 2024-04-25T11:43:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.455+0000 7f8d065e2740 1 -- 172.21.15.83:0/1890480074 shutdown_connections 2024-04-25T11:43:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.455+0000 7f8d065e2740 1 --2- 172.21.15.83:0/1890480074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d9710 0x556a326e1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.455+0000 7f8d065e2740 1 --2- 172.21.15.83:0/1890480074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d8d50 0x556a326e9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.455+0000 7f8d065e2740 1 --2- 172.21.15.83:0/1890480074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d7b80 0x556a326edc50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.455+0000 7f8d065e2740 1 -- 172.21.15.83:0/1890480074 >> 172.21.15.83:0/1890480074 conn(0x556a326d3070 msgr2=0x556a326ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.455+0000 7f8d065e2740 1 -- 172.21.15.83:0/1890480074 shutdown_connections 2024-04-25T11:43:09.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.455+0000 7f8d065e2740 1 -- 172.21.15.83:0/1890480074 wait complete. 2024-04-25T11:43:09.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.455+0000 7f8d065e2740 1 -- start start 2024-04-25T11:43:09.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.455+0000 7f8d065e2740 1 Processor -- start 2024-04-25T11:43:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d065e2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d7b80 0x556a326e4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d065e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d8d50 0x556a326e4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d065e2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d9710 0x556a32752e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d065e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556a326f0160 con 0x556a326d8d50 2024-04-25T11:43:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d065e2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x556a326efa90 con 0x556a326d9710 2024-04-25T11:43:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d065e2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556a326f0830 con 0x556a326d7b80 2024-04-25T11:43:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d065e2740 10 client.0 _ll_register_callbacks cb 0x7f8cf4014280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d00e6b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d8d50 0x556a326e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d00e6b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d8d50 0x556a326e4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33092/0 (socket says 172.21.15.83:33092) 2024-04-25T11:43:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d00e6b640 1 -- 172.21.15.83:0/2375979597 learned_addr learned my addr 172.21.15.83:0/2375979597 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d01e6d640 1 --2- 172.21.15.83:0/2375979597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d9710 0x556a32752e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d0166c640 1 --2- 172.21.15.83:0/2375979597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d7b80 0x556a326e4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d00e6b640 1 -- 172.21.15.83:0/2375979597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d7b80 msgr2=0x556a326e4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d00e6b640 1 --2- 172.21.15.83:0/2375979597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d7b80 0x556a326e4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d00e6b640 1 -- 172.21.15.83:0/2375979597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d9710 msgr2=0x556a32752e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d00e6b640 1 --2- 172.21.15.83:0/2375979597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d9710 0x556a32752e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d00e6b640 1 -- 172.21.15.83:0/2375979597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a32753350 con 0x556a326d8d50 2024-04-25T11:43:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d01e6d640 1 --2- 172.21.15.83:0/2375979597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d9710 0x556a32752e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d00e6b640 1 --2- 172.21.15.83:0/2375979597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d8d50 0x556a326e4840 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc06b0f0 tx=0x7f8cec00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7e473d6ef57135 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8ce4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8ce4ff9640 1 -- 172.21.15.83:0/2375979597 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec026470 con 0x556a326d8d50 2024-04-25T11:43:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8ce4ff9640 1 -- 172.21.15.83:0/2375979597 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f8cec026610 con 0x556a326d8d50 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8d065e2740 1 -- 172.21.15.83:0/2375979597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cf804cdc0 con 0x556a326d8d50 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.456+0000 7f8ce4ff9640 1 -- 172.21.15.83:0/2375979597 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8cec026810 con 0x556a326d8d50 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.457+0000 7f8d065e2740 10 client.15138 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.457+0000 7f8d065e2740 1 -- 172.21.15.83:0/2375979597 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556a32754680 con 0x556a326d8d50 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.457+0000 7f8ce4ff9640 1 -- 172.21.15.83:0/2375979597 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cf804cdc0 con 0x556a326d8d50 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.457+0000 7f8ce4ff9640 1 -- 172.21.15.83:0/2375979597 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f8cec01cb70 con 0x556a326d8d50 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.457+0000 7f8d065e2740 10 client.15138 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.457+0000 7f8d065e2740 10 client.15138 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.457+0000 7f8d065e2740 20 client.15138 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.457+0000 7f8d065e2740 1 --2- 172.21.15.83:0/2375979597 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x556a32759270 0x556a32755d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.457+0000 7f8d065e2740 4 client.15138 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.457+0000 7f8d065e2740 1 -- 172.21.15.83:0/2375979597 --> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] -- command(tid 0: {"prefix": "status"}) -- 0x556a326444e0 con 0x556a32759270 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.457+0000 7f8d0166c640 1 --2- 172.21.15.83:0/2375979597 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x556a32759270 0x556a32755d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8ce4ff9640 10 client.15138 ms_handle_connect on v2:172.21.15.94:6838/1461078471 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8d0166c640 1 --2- 172.21.15.83:0/2375979597 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x556a32759270 0x556a32755d80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0964573f0fd31db server_cookie=8dd6b10d5cab7370 in_seq=0 out_seq=0 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8ce4ff9640 1 -- 172.21.15.83:0/2375979597 <== mds.0 v2:172.21.15.94:6838/1461078471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556a326444e0 con 0x556a32759270 2024-04-25T11:43:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8ce4ff9640 10 client.15138 handle_command_reply: tid=0 2024-04-25T11:43:09.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8d065e2740 1 client.15138 shutdown 2024-04-25T11:43:09.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8d065e2740 1 -- 172.21.15.83:0/2375979597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d8d50 msgr2=0x556a326e4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8d065e2740 1 --2- 172.21.15.83:0/2375979597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d8d50 0x556a326e4840 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc06b0f0 tx=0x7f8cec00b8c0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8d065e2740 1 -- 172.21.15.83:0/2375979597 shutdown_connections 2024-04-25T11:43:09.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8d065e2740 1 --2- 172.21.15.83:0/2375979597 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x556a32759270 0x556a32755d80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8d065e2740 1 --2- 172.21.15.83:0/2375979597 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a326d8d50 0x556a326e4840 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8d065e2740 1 --2- 172.21.15.83:0/2375979597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556a326d9710 0x556a32752e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8ce4ff9640 0 client.15138 ms_handle_reset on v2:172.21.15.94:6838/1461078471 2024-04-25T11:43:09.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8d065e2740 1 --2- 172.21.15.83:0/2375979597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a326d7b80 0x556a326e4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8d065e2740 1 -- 172.21.15.83:0/2375979597 >> 172.21.15.83:0/2375979597 conn(0x556a326d3070 msgr2=0x556a326e1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8d065e2740 1 -- 172.21.15.83:0/2375979597 shutdown_connections 2024-04-25T11:43:09.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8d065e2740 1 -- 172.21.15.83:0/2375979597 wait complete. 2024-04-25T11:43:09.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.458+0000 7f8d065e2740 20 client.15138 trim_cache size 0 max 16384 2024-04-25T11:43:09.451 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:43:09.451 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "7fcf54c6-02f7-11ef-bc93-c7b262605968", 2024-04-25T11:43:09.451 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-25T11:43:09.451 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24479, 2024-04-25T11:43:09.451 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T11:43:09.451 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T11:43:09.451 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T11:43:09.451 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 84.018777634000003, 2024-04-25T11:43:09.451 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-04-25T11:43:09.451 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-25T11:43:09.451 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T11:43:09.451 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 89.386623087999993 2024-04-25T11:43:09.451 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:43:09.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.459+0000 7f8d038d0640 1 -- 172.21.15.83:0/603222754 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8ce80a3cc0 msgr2=0x7f8ce80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.459+0000 7f8d038d0640 1 --2- 172.21.15.83:0/603222754 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8ce80a3cc0 0x7f8ce80a6180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc1020c0 tx=0x7f8cec00b930 comp rx=0 tx=0).stop 2024-04-25T11:43:09.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.459+0000 7f8d038d0640 1 -- 172.21.15.83:0/603222754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc107c90 msgr2=0x7f8cfc100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:09.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.459+0000 7f8d038d0640 1 --2- 172.21.15.83:0/603222754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc107c90 0x7f8cfc100f20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400cad0 tx=0x7f8cf400cf90 comp rx=0 tx=0).stop 2024-04-25T11:43:09.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.459+0000 7f8d038d0640 1 -- 172.21.15.83:0/603222754 shutdown_connections 2024-04-25T11:43:09.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.459+0000 7f8d038d0640 1 --2- 172.21.15.83:0/603222754 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f8ce80a3cc0 0x7f8ce80a6180 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.459+0000 7f8d038d0640 1 --2- 172.21.15.83:0/603222754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cfc107c90 0x7f8cfc100f20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.459+0000 7f8d038d0640 1 --2- 172.21.15.83:0/603222754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8cfc105360 0x7f8cfc1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.459+0000 7f8d038d0640 1 --2- 172.21.15.83:0/603222754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc102a30 0x7f8cfc102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:09.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.459+0000 7f8d038d0640 1 -- 172.21.15.83:0/603222754 >> 172.21.15.83:0/603222754 conn(0x7f8cfc0fc440 msgr2=0x7f8cfc103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:09.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.459+0000 7f8d038d0640 1 -- 172.21.15.83:0/603222754 shutdown_connections 2024-04-25T11:43:09.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:09.459+0000 7f8d038d0640 1 -- 172.21.15.83:0/603222754 wait complete. 2024-04-25T11:43:09.456 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T11:43:09.456 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T11:43:09.464 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T11:43:09.465 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T11:43:09.699 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:43:11.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:11 smithi083 ceph-mon[29330]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T11:43:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.241+0000 7f0a0f577700 1 -- 172.21.15.83:0/2432897484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a081019a0 msgr2=0x7f0a08101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.241+0000 7f0a0f577700 1 --2- 172.21.15.83:0/2432897484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a081019a0 0x7f0a08101dc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000b0d0 tx=0x7f0a0000b3e0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.241+0000 7f0a0f577700 1 -- 172.21.15.83:0/2432897484 shutdown_connections 2024-04-25T11:43:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.241+0000 7f0a0f577700 1 --2- 172.21.15.83:0/2432897484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a08103540 0x7f0a08109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.241+0000 7f0a0f577700 1 --2- 172.21.15.83:0/2432897484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a081019a0 0x7f0a08101dc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.241+0000 7f0a0f577700 1 --2- 172.21.15.83:0/2432897484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a08102ba0 0x7f0a08103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.241+0000 7f0a0f577700 1 -- 172.21.15.83:0/2432897484 >> 172.21.15.83:0/2432897484 conn(0x7f0a080fcf40 msgr2=0x7f0a080ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:11.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.241+0000 7f0a0f577700 1 -- 172.21.15.83:0/2432897484 shutdown_connections 2024-04-25T11:43:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.241+0000 7f0a0f577700 1 -- 172.21.15.83:0/2432897484 wait complete. 2024-04-25T11:43:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.241+0000 7f0a0f577700 1 Processor -- start 2024-04-25T11:43:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0f577700 1 -- start start 2024-04-25T11:43:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0f577700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a081019a0 0x7f0a08105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0f577700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a08102ba0 0x7f0a08103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0f577700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a08103540 0x7f0a081044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0f577700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0810d410 con 0x7f0a081019a0 2024-04-25T11:43:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0f577700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0810d290 con 0x7f0a08102ba0 2024-04-25T11:43:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0f577700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0810d590 con 0x7f0a08103540 2024-04-25T11:43:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0d313700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a081019a0 0x7f0a08105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0d313700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a081019a0 0x7f0a08105930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33102/0 (socket says 172.21.15.83:33102) 2024-04-25T11:43:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0d313700 1 -- 172.21.15.83:0/842162971 learned_addr learned my addr 172.21.15.83:0/842162971 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0cb12700 1 --2- 172.21.15.83:0/842162971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a08102ba0 0x7f0a08103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0db14700 1 --2- 172.21.15.83:0/842162971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a08103540 0x7f0a081044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0d313700 1 -- 172.21.15.83:0/842162971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a08103540 msgr2=0x7f0a081044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0d313700 1 --2- 172.21.15.83:0/842162971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a08103540 0x7f0a081044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0d313700 1 -- 172.21.15.83:0/842162971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a08102ba0 msgr2=0x7f0a08103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0d313700 1 --2- 172.21.15.83:0/842162971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a08102ba0 0x7f0a08103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0d313700 1 -- 172.21.15.83:0/842162971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a08104d70 con 0x7f0a081019a0 2024-04-25T11:43:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0db14700 1 --2- 172.21.15.83:0/842162971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a08103540 0x7f0a081044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a0d313700 1 --2- 172.21.15.83:0/842162971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a081019a0 0x7f0a08105930 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000d010 tx=0x7f0a00007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.242+0000 7f0a067fc700 1 -- 172.21.15.83:0/842162971 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00003c00 con 0x7f0a081019a0 2024-04-25T11:43:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.243+0000 7f0a0f577700 1 -- 172.21.15.83:0/842162971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a081a2d20 con 0x7f0a081019a0 2024-04-25T11:43:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.243+0000 7f0a067fc700 1 -- 172.21.15.83:0/842162971 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a00003da0 con 0x7f0a081019a0 2024-04-25T11:43:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.243+0000 7f0a067fc700 1 -- 172.21.15.83:0/842162971 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0001b840 con 0x7f0a081019a0 2024-04-25T11:43:11.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.243+0000 7f0a0f577700 1 -- 172.21.15.83:0/842162971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a081a3170 con 0x7f0a081019a0 2024-04-25T11:43:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.243+0000 7f0a0f577700 1 -- 172.21.15.83:0/842162971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a08069730 con 0x7f0a081019a0 2024-04-25T11:43:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.244+0000 7f0a067fc700 1 -- 172.21.15.83:0/842162971 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0a00026450 con 0x7f0a081019a0 2024-04-25T11:43:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.245+0000 7f0a067fc700 1 --2- 172.21.15.83:0/842162971 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f09f00a3cd0 0x7f09f00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.245+0000 7f0a0cb12700 1 --2- 172.21.15.83:0/842162971 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f09f00a3cd0 0x7f09f00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.245+0000 7f0a067fc700 1 -- 172.21.15.83:0/842162971 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f0a0000f020 con 0x7f0a081019a0 2024-04-25T11:43:11.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.245+0000 7f0a0cb12700 1 --2- 172.21.15.83:0/842162971 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f09f00a3cd0 0x7f09f00a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00ac50 tx=0x7f09fc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:11.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.250+0000 7f0a067fc700 1 -- 172.21.15.83:0/842162971 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a000074e0 con 0x7f0a081019a0 2024-04-25T11:43:11.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:11 smithi094 ceph-mon[33032]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T11:43:11.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:11 smithi111 ceph-mon[32784]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T11:43:11.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.467+0000 7f0a0f577700 1 -- 172.21.15.83:0/842162971 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f0a080ff160 con 0x7f09f00a3cd0 2024-04-25T11:43:11.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.468+0000 7f0a067fc700 1 -- 172.21.15.83:0/842162971 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f0a080ff160 con 0x7f09f00a3cd0 2024-04-25T11:43:11.455 INFO:teuthology.orchestra.run.smithi083.stdout:Backend: cephadm 2024-04-25T11:43:11.455 INFO:teuthology.orchestra.run.smithi083.stdout:Available: Yes 2024-04-25T11:43:11.455 INFO:teuthology.orchestra.run.smithi083.stdout:Paused: No 2024-04-25T11:43:11.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.470+0000 7f0a0f577700 1 -- 172.21.15.83:0/842162971 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f09f00a3cd0 msgr2=0x7f09f00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:11.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.471+0000 7f0a0f577700 1 --2- 172.21.15.83:0/842162971 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f09f00a3cd0 0x7f09f00a6190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00ac50 tx=0x7f09fc00a380 comp rx=0 tx=0).stop 2024-04-25T11:43:11.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.471+0000 7f0a0f577700 1 -- 172.21.15.83:0/842162971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a081019a0 msgr2=0x7f0a08105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:11.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.471+0000 7f0a0f577700 1 --2- 172.21.15.83:0/842162971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a081019a0 0x7f0a08105930 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000d010 tx=0x7f0a00007a20 comp rx=0 tx=0).stop 2024-04-25T11:43:11.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.471+0000 7f0a0f577700 1 -- 172.21.15.83:0/842162971 shutdown_connections 2024-04-25T11:43:11.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.471+0000 7f0a0f577700 1 --2- 172.21.15.83:0/842162971 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f09f00a3cd0 0x7f09f00a6190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.471+0000 7f0a0f577700 1 --2- 172.21.15.83:0/842162971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0a08103540 0x7f0a081044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.471+0000 7f0a0f577700 1 --2- 172.21.15.83:0/842162971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0a081019a0 0x7f0a08105930 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.471+0000 7f0a0f577700 1 --2- 172.21.15.83:0/842162971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a08102ba0 0x7f0a08103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:11.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.471+0000 7f0a0f577700 1 -- 172.21.15.83:0/842162971 >> 172.21.15.83:0/842162971 conn(0x7f0a080fcf40 msgr2=0x7f0a080fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:11.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.471+0000 7f0a0f577700 1 -- 172.21.15.83:0/842162971 shutdown_connections 2024-04-25T11:43:11.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:11.471+0000 7f0a0f577700 1 -- 172.21.15.83:0/842162971 wait complete. 2024-04-25T11:43:12.122 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T11:43:12.366 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:43:13.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:13 smithi094 ceph-mon[33032]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:13.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:13 smithi094 ceph-mon[33032]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:43:13.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:13 smithi111 ceph-mon[32784]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:13.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:13 smithi111 ceph-mon[32784]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:43:13.361 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:13 smithi083 ceph-mon[29330]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:13.361 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:13 smithi083 ceph-mon[29330]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-25T11:43:13.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 -- 172.21.15.83:0/1686489315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b0101b20 msgr2=0x7fa9b0101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:13.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 --2- 172.21.15.83:0/1686489315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b0101b20 0x7fa9b0101fa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa99c009990 tx=0x7fa99c009ca0 comp rx=0 tx=0).stop 2024-04-25T11:43:13.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 -- 172.21.15.83:0/1686489315 shutdown_connections 2024-04-25T11:43:13.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 --2- 172.21.15.83:0/1686489315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b01024e0 0x7fa9b0102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:13.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 --2- 172.21.15.83:0/1686489315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9b0100db0 0x7fa9b01011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:13.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 --2- 172.21.15.83:0/1686489315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b0101b20 0x7fa9b0101fa0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:13.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 -- 172.21.15.83:0/1686489315 >> 172.21.15.83:0/1686489315 conn(0x7fa9b00fc710 msgr2=0x7fa9b00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:13.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 -- 172.21.15.83:0/1686489315 shutdown_connections 2024-04-25T11:43:13.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 -- 172.21.15.83:0/1686489315 wait complete. 2024-04-25T11:43:13.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 Processor -- start 2024-04-25T11:43:13.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 -- start start 2024-04-25T11:43:13.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b0100db0 0x7fa9b019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:13.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b0101b20 0x7fa9b019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:13.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9b01024e0 0x7fa9b01a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:13.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0110820 con 0x7fa9b0101b20 2024-04-25T11:43:13.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b01106a0 con 0x7fa9b0100db0 2024-04-25T11:43:13.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9b6f91700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b01109a0 con 0x7fa9b01024e0 2024-04-25T11:43:13.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9affff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b0101b20 0x7fa9b019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:13.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.964+0000 7fa9affff700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b0101b20 0x7fa9b019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54662/0 (socket says 172.21.15.83:54662) 2024-04-25T11:43:13.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.965+0000 7fa9affff700 1 -- 172.21.15.83:0/323421896 learned_addr learned my addr 172.21.15.83:0/323421896 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:13.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.965+0000 7fa9b4d2d700 1 --2- 172.21.15.83:0/323421896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b0100db0 0x7fa9b019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:13.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.965+0000 7fa9affff700 1 -- 172.21.15.83:0/323421896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9b01024e0 msgr2=0x7fa9b01a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:13.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.965+0000 7fa9affff700 1 --2- 172.21.15.83:0/323421896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9b01024e0 0x7fa9b01a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:13.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.965+0000 7fa9affff700 1 -- 172.21.15.83:0/323421896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b0100db0 msgr2=0x7fa9b019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:13.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.965+0000 7fa9affff700 1 --2- 172.21.15.83:0/323421896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b0100db0 0x7fa9b019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:13.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.965+0000 7fa9affff700 1 -- 172.21.15.83:0/323421896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b01a2a90 con 0x7fa9b0101b20 2024-04-25T11:43:13.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.965+0000 7fa9b4d2d700 1 --2- 172.21.15.83:0/323421896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b0100db0 0x7fa9b019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:13.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.965+0000 7fa9affff700 1 --2- 172.21.15.83:0/323421896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b0101b20 0x7fa9b019b370 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0036f0 tx=0x7fa99c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:13.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.965+0000 7fa9adffb700 1 -- 172.21.15.83:0/323421896 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c027070 con 0x7fa9b0101b20 2024-04-25T11:43:13.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.965+0000 7fa9adffb700 1 -- 172.21.15.83:0/323421896 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa99c004180 con 0x7fa9b0101b20 2024-04-25T11:43:13.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.965+0000 7fa9adffb700 1 -- 172.21.15.83:0/323421896 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99c022de0 con 0x7fa9b0101b20 2024-04-25T11:43:13.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.965+0000 7fa9b6f91700 1 -- 172.21.15.83:0/323421896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b01a2d20 con 0x7fa9b0101b20 2024-04-25T11:43:13.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.965+0000 7fa9b6f91700 1 -- 172.21.15.83:0/323421896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b01a31a0 con 0x7fa9b0101b20 2024-04-25T11:43:13.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.966+0000 7fa9b6f91700 1 -- 172.21.15.83:0/323421896 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b002df80 con 0x7fa9b0101b20 2024-04-25T11:43:13.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.967+0000 7fa9adffb700 1 -- 172.21.15.83:0/323421896 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa99c02a070 con 0x7fa9b0101b20 2024-04-25T11:43:13.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.968+0000 7fa9adffb700 1 --2- 172.21.15.83:0/323421896 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa9a00a3cd0 0x7fa9a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:13.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.968+0000 7fa9b4d2d700 1 --2- 172.21.15.83:0/323421896 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa9a00a3cd0 0x7fa9a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:13.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.968+0000 7fa9adffb700 1 -- 172.21.15.83:0/323421896 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fa99c0d1570 con 0x7fa9b0101b20 2024-04-25T11:43:13.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.969+0000 7fa9b4d2d700 1 --2- 172.21.15.83:0/323421896 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa9a00a3cd0 0x7fa9a00a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4007900 tx=0x7fa9a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:13.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:13.973+0000 7fa9adffb700 1 -- 172.21.15.83:0/323421896 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa99c05eff0 con 0x7fa9b0101b20 2024-04-25T11:43:14.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:14.189+0000 7fa9b6f91700 1 -- 172.21.15.83:0/323421896 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fa9b0062e80 con 0x7fa9a00a3cd0 2024-04-25T11:43:14.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:14.203+0000 7fa9adffb700 1 -- 172.21.15.83:0/323421896 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fa9b0062e80 con 0x7fa9a00a3cd0 2024-04-25T11:43:14.189 INFO:teuthology.orchestra.run.smithi083.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T11:43:14.189 INFO:teuthology.orchestra.run.smithi083.stdout:mds.a smithi083 running (97s) 17s ago 96s 18.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6680a114ecc0 2024-04-25T11:43:14.189 INFO:teuthology.orchestra.run.smithi083.stdout:mds.b smithi094 running (94s) 28s ago 94s 25.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 abf805b5081f 2024-04-25T11:43:14.189 INFO:teuthology.orchestra.run.smithi083.stdout:mds.c smithi111 running (108s) 28s ago 107s 16.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 b3c289675af1 2024-04-25T11:43:14.190 INFO:teuthology.orchestra.run.smithi083.stdout:mds.d smithi083 running (116s) 17s ago 116s 16.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 5b29031497ff 2024-04-25T11:43:14.190 INFO:teuthology.orchestra.run.smithi083.stdout:mds.e smithi094 running (113s) 28s ago 113s 22.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 5e04a0850058 2024-04-25T11:43:14.190 INFO:teuthology.orchestra.run.smithi083.stdout:mds.f smithi111 running (111s) 28s ago 110s 18.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 16d4d3696cf6 2024-04-25T11:43:14.190 INFO:teuthology.orchestra.run.smithi083.stdout:mds.g smithi083 running (100s) 17s ago 99s 19.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 1a1f96ec98c9 2024-04-25T11:43:14.190 INFO:teuthology.orchestra.run.smithi083.stdout:mds.h smithi094 running (105s) 28s ago 105s 19.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 0ad77a914ff4 2024-04-25T11:43:14.190 INFO:teuthology.orchestra.run.smithi083.stdout:mds.i smithi111 running (102s) 28s ago 102s 17.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ab2aba19559d 2024-04-25T11:43:14.190 INFO:teuthology.orchestra.run.smithi083.stdout:mds.j smithi083 running (119s) 17s ago 118s 16.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 fb95b994b5c6 2024-04-25T11:43:14.190 INFO:teuthology.orchestra.run.smithi083.stdout:mds.k smithi094 running (2m) 28s ago 2m 16.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 be554274b8ae 2024-04-25T11:43:14.190 INFO:teuthology.orchestra.run.smithi083.stdout:mds.l smithi111 running (2m) 28s ago 2m 15.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ee8cf3d49253 2024-04-25T11:43:14.190 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.x smithi083 *:9283,8765 running (9m) 17s ago 9m 514M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6c82361a54d3 2024-04-25T11:43:14.190 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.y smithi094 *:8443,8765 running (7m) 28s ago 7m 455M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6b433dab8161 2024-04-25T11:43:14.190 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.z smithi111 *:8443,8765 running (7m) 28s ago 7m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3983775f5066 2024-04-25T11:43:14.190 INFO:teuthology.orchestra.run.smithi083.stdout:mon.a smithi083 running (9m) 17s ago 9m 48.2M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 b35e79aacc4c 2024-04-25T11:43:14.191 INFO:teuthology.orchestra.run.smithi083.stdout:mon.b smithi094 running (7m) 28s ago 7m 39.2M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 5643b9010621 2024-04-25T11:43:14.191 INFO:teuthology.orchestra.run.smithi083.stdout:mon.c smithi111 running (7m) 28s ago 7m 42.4M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 a736b2f2b4a6 2024-04-25T11:43:14.191 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0 smithi083 running (6m) 17s ago 6m 48.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 027c79d7f018 2024-04-25T11:43:14.191 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1 smithi094 running (6m) 28s ago 6m 51.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 4f664b7fa4f5 2024-04-25T11:43:14.191 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2 smithi111 running (6m) 28s ago 6m 50.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 73cc4e57e47b 2024-04-25T11:43:14.191 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3 smithi083 running (5m) 17s ago 5m 57.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 29afb0f84455 2024-04-25T11:43:14.191 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 smithi094 running (5m) 28s ago 5m 45.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 f1a32c4b5cc0 2024-04-25T11:43:14.191 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5 smithi111 running (4m) 28s ago 4m 47.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 9b336e1b8252 2024-04-25T11:43:14.191 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6 smithi083 running (4m) 17s ago 4m 53.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 9f33e0db3c6d 2024-04-25T11:43:14.191 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7 smithi094 running (4m) 28s ago 4m 54.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 fff1578a276b 2024-04-25T11:43:14.191 INFO:teuthology.orchestra.run.smithi083.stdout:osd.8 smithi111 running (3m) 28s ago 3m 47.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 3df1d899de3a 2024-04-25T11:43:14.191 INFO:teuthology.orchestra.run.smithi083.stdout:osd.9 smithi083 running (3m) 17s ago 3m 52.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 321290513c17 2024-04-25T11:43:14.191 INFO:teuthology.orchestra.run.smithi083.stdout:osd.10 smithi094 running (2m) 28s ago 2m 55.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b26b8291c60a 2024-04-25T11:43:14.192 INFO:teuthology.orchestra.run.smithi083.stdout:osd.11 smithi111 running (2m) 28s ago 2m 43.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 d56a4b43abfd 2024-04-25T11:43:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:14.205+0000 7fa9b6f91700 1 -- 172.21.15.83:0/323421896 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa9a00a3cd0 msgr2=0x7fa9a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:14.205+0000 7fa9b6f91700 1 --2- 172.21.15.83:0/323421896 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa9a00a3cd0 0x7fa9a00a6190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4007900 tx=0x7fa9a4008040 comp rx=0 tx=0).stop 2024-04-25T11:43:14.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:14.205+0000 7fa9b6f91700 1 -- 172.21.15.83:0/323421896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b0101b20 msgr2=0x7fa9b019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:14.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:14.205+0000 7fa9b6f91700 1 --2- 172.21.15.83:0/323421896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b0101b20 0x7fa9b019b370 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0036f0 tx=0x7fa99c0047b0 comp rx=0 tx=0).stop 2024-04-25T11:43:14.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:14.205+0000 7fa9b6f91700 1 -- 172.21.15.83:0/323421896 shutdown_connections 2024-04-25T11:43:14.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:14.205+0000 7fa9b6f91700 1 --2- 172.21.15.83:0/323421896 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fa9a00a3cd0 0x7fa9a00a6190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:14.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:14.205+0000 7fa9b6f91700 1 --2- 172.21.15.83:0/323421896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9b01024e0 0x7fa9b01a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:14.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:14.205+0000 7fa9b6f91700 1 --2- 172.21.15.83:0/323421896 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9b0100db0 0x7fa9b019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:14.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:14.205+0000 7fa9b6f91700 1 --2- 172.21.15.83:0/323421896 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9b0101b20 0x7fa9b019b370 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:14.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:14.205+0000 7fa9b6f91700 1 -- 172.21.15.83:0/323421896 >> 172.21.15.83:0/323421896 conn(0x7fa9b00fc710 msgr2=0x7fa9b00fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:14.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:14.205+0000 7fa9b6f91700 1 -- 172.21.15.83:0/323421896 shutdown_connections 2024-04-25T11:43:14.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:14.205+0000 7fa9b6f91700 1 -- 172.21.15.83:0/323421896 wait complete. 2024-04-25T11:43:14.855 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T11:43:15.098 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:43:15.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:15 smithi083 ceph-mon[29330]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T11:43:15.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:15 smithi094 ceph-mon[33032]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T11:43:15.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:15 smithi111 ceph-mon[32784]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-25T11:43:16.270 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:16 smithi083 ceph-mon[29330]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:16.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:16 smithi094 ceph-mon[33032]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:16.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:16 smithi111 ceph-mon[32784]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.501+0000 7f43f5861700 1 -- 172.21.15.83:0/3174607069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f01021c0 msgr2=0x7f43f01045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.501+0000 7f43f5861700 1 --2- 172.21.15.83:0/3174607069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f01021c0 0x7f43f01045b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009990 tx=0x7f43e0009ca0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.502+0000 7f43f5861700 1 -- 172.21.15.83:0/3174607069 shutdown_connections 2024-04-25T11:43:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.502+0000 7f43f5861700 1 --2- 172.21.15.83:0/3174607069 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43f0107630 0x7f43f0109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.502+0000 7f43f5861700 1 --2- 172.21.15.83:0/3174607069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f01021c0 0x7f43f01045b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.502+0000 7f43f5861700 1 --2- 172.21.15.83:0/3174607069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43f0104b80 0x7f43f0106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.502+0000 7f43f5861700 1 -- 172.21.15.83:0/3174607069 >> 172.21.15.83:0/3174607069 conn(0x7f43f00fc290 msgr2=0x7f43f00fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.502+0000 7f43f5861700 1 -- 172.21.15.83:0/3174607069 shutdown_connections 2024-04-25T11:43:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.502+0000 7f43f5861700 1 -- 172.21.15.83:0/3174607069 wait complete. 2024-04-25T11:43:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.502+0000 7f43f5861700 1 Processor -- start 2024-04-25T11:43:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.502+0000 7f43f5861700 1 -- start start 2024-04-25T11:43:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.502+0000 7f43f5861700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43f01021c0 0x7f43f0198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.502+0000 7f43f5861700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f0104b80 0x7f43f0199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.502+0000 7f43f5861700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43f0107630 0x7f43f01a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.502+0000 7f43f5861700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43f010d480 con 0x7f43f0104b80 2024-04-25T11:43:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.502+0000 7f43f5861700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f43f010d300 con 0x7f43f0107630 2024-04-25T11:43:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.502+0000 7f43f5861700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f43f010d600 con 0x7f43f01021c0 2024-04-25T11:43:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43ee7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f0104b80 0x7f43f0199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43ee7fc700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f0104b80 0x7f43f0199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54676/0 (socket says 172.21.15.83:54676) 2024-04-25T11:43:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43ee7fc700 1 -- 172.21.15.83:0/4037903302 learned_addr learned my addr 172.21.15.83:0/4037903302 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:16.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43ef7fe700 1 --2- 172.21.15.83:0/4037903302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43f0107630 0x7f43f01a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:16.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43eeffd700 1 --2- 172.21.15.83:0/4037903302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43f01021c0 0x7f43f0198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:16.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43ee7fc700 1 -- 172.21.15.83:0/4037903302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43f01021c0 msgr2=0x7f43f0198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:16.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43ee7fc700 1 --2- 172.21.15.83:0/4037903302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43f01021c0 0x7f43f0198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43ee7fc700 1 -- 172.21.15.83:0/4037903302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43f0107630 msgr2=0x7f43f01a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:16.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43ee7fc700 1 --2- 172.21.15.83:0/4037903302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43f0107630 0x7f43f01a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43ee7fc700 1 -- 172.21.15.83:0/4037903302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f01a0870 con 0x7f43f0104b80 2024-04-25T11:43:16.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43ee7fc700 1 --2- 172.21.15.83:0/4037903302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f0104b80 0x7f43f0199160 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f43e400b530 tx=0x7f43e400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:16.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43f485f700 1 -- 172.21.15.83:0/4037903302 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4015020 con 0x7f43f0104b80 2024-04-25T11:43:16.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43f5861700 1 -- 172.21.15.83:0/4037903302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f01a0b60 con 0x7f43f0104b80 2024-04-25T11:43:16.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43f485f700 1 -- 172.21.15.83:0/4037903302 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43e4011ce0 con 0x7f43f0104b80 2024-04-25T11:43:16.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43f485f700 1 -- 172.21.15.83:0/4037903302 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4010dd0 con 0x7f43f0104b80 2024-04-25T11:43:16.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.503+0000 7f43f5861700 1 -- 172.21.15.83:0/4037903302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f01a10a0 con 0x7f43f0104b80 2024-04-25T11:43:16.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.504+0000 7f43f5861700 1 -- 172.21.15.83:0/4037903302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f0069730 con 0x7f43f0104b80 2024-04-25T11:43:16.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.506+0000 7f43f485f700 1 -- 172.21.15.83:0/4037903302 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f43e4018020 con 0x7f43f0104b80 2024-04-25T11:43:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.506+0000 7f43f485f700 1 --2- 172.21.15.83:0/4037903302 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f43d80a3cd0 0x7f43d80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.506+0000 7f43eeffd700 1 --2- 172.21.15.83:0/4037903302 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f43d80a3cd0 0x7f43d80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.506+0000 7f43f485f700 1 -- 172.21.15.83:0/4037903302 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f43e40d00e0 con 0x7f43f0104b80 2024-04-25T11:43:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.507+0000 7f43eeffd700 1 --2- 172.21.15.83:0/4037903302 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f43d80a3cd0 0x7f43d80a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f43f019a100 tx=0x7f43e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.510+0000 7f43f485f700 1 -- 172.21.15.83:0/4037903302 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43e401f020 con 0x7f43f0104b80 2024-04-25T11:43:16.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.721+0000 7f43f5861700 1 -- 172.21.15.83:0/4037903302 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f43f00fd680 con 0x7f43d80a3cd0 2024-04-25T11:43:16.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.727+0000 7f43f485f700 1 -- 172.21.15.83:0/4037903302 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f43f00fd680 con 0x7f43d80a3cd0 2024-04-25T11:43:16.713 INFO:teuthology.orchestra.run.smithi083.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T11:43:16.713 INFO:teuthology.orchestra.run.smithi083.stdout:mds.all 12/12 31s ago 2m smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi094=b;smithi094=e;smithi094=h;smithi094=k;smithi111=c;smithi111=f;smithi111=i;smithi111=l;count:12 2024-04-25T11:43:16.713 INFO:teuthology.orchestra.run.smithi083.stdout:mgr 3/3 31s ago 7m smithi083=x;smithi094=y;smithi111=z;count:3 2024-04-25T11:43:16.713 INFO:teuthology.orchestra.run.smithi083.stdout:mon 3/3 31s ago 7m smithi083:172.21.15.83=a;smithi094:172.21.15.94=b;smithi111:172.21.15.111=c;count:3 2024-04-25T11:43:16.714 INFO:teuthology.orchestra.run.smithi083.stdout:osd 12 31s ago - 2024-04-25T11:43:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.729+0000 7f43f5861700 1 -- 172.21.15.83:0/4037903302 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f43d80a3cd0 msgr2=0x7f43d80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.729+0000 7f43f5861700 1 --2- 172.21.15.83:0/4037903302 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f43d80a3cd0 0x7f43d80a6190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f43f019a100 tx=0x7f43e0005fb0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.729+0000 7f43f5861700 1 -- 172.21.15.83:0/4037903302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f0104b80 msgr2=0x7f43f0199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.729+0000 7f43f5861700 1 --2- 172.21.15.83:0/4037903302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f0104b80 0x7f43f0199160 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f43e400b530 tx=0x7f43e400b840 comp rx=0 tx=0).stop 2024-04-25T11:43:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.729+0000 7f43f5861700 1 -- 172.21.15.83:0/4037903302 shutdown_connections 2024-04-25T11:43:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.729+0000 7f43f5861700 1 --2- 172.21.15.83:0/4037903302 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f43d80a3cd0 0x7f43d80a6190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.729+0000 7f43f5861700 1 --2- 172.21.15.83:0/4037903302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43f0107630 0x7f43f01a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.729+0000 7f43f5861700 1 --2- 172.21.15.83:0/4037903302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43f01021c0 0x7f43f0198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.729+0000 7f43f5861700 1 --2- 172.21.15.83:0/4037903302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f43f0104b80 0x7f43f0199160 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:16.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.729+0000 7f43f5861700 1 -- 172.21.15.83:0/4037903302 >> 172.21.15.83:0/4037903302 conn(0x7f43f00fc290 msgr2=0x7f43f0101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:16.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.730+0000 7f43f5861700 1 -- 172.21.15.83:0/4037903302 shutdown_connections 2024-04-25T11:43:16.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:16.730+0000 7f43f5861700 1 -- 172.21.15.83:0/4037903302 wait complete. 2024-04-25T11:43:17.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:17 smithi083 ceph-mon[29330]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T11:43:17.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:17 smithi094 ceph-mon[33032]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T11:43:17.287 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T11:43:17.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:17 smithi111 ceph-mon[32784]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-04-25T11:43:17.525 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:43:18.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:18 smithi094 ceph-mon[33032]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:18.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:18 smithi111 ceph-mon[32784]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:18.349 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:18 smithi083 ceph-mon[29330]: from='client.15156 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:19.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.160+0000 7f2524e0e700 1 -- 172.21.15.83:0/1858320177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2520104660 msgr2=0x7f252010af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.160+0000 7f2524e0e700 1 --2- 172.21.15.83:0/1858320177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2520104660 0x7f252010af10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f251800b0d0 tx=0x7f251800b3e0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.160+0000 7f2524e0e700 1 -- 172.21.15.83:0/1858320177 shutdown_connections 2024-04-25T11:43:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.160+0000 7f2524e0e700 1 --2- 172.21.15.83:0/1858320177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2520104660 0x7f252010af10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.160+0000 7f2524e0e700 1 --2- 172.21.15.83:0/1858320177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2520102ac0 0x7f2520102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.160+0000 7f2524e0e700 1 --2- 172.21.15.83:0/1858320177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2520103cc0 0x7f2520104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.160+0000 7f2524e0e700 1 -- 172.21.15.83:0/1858320177 >> 172.21.15.83:0/1858320177 conn(0x7f25200fe060 msgr2=0x7f25201004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.160+0000 7f2524e0e700 1 -- 172.21.15.83:0/1858320177 shutdown_connections 2024-04-25T11:43:19.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.160+0000 7f2524e0e700 1 -- 172.21.15.83:0/1858320177 wait complete. 2024-04-25T11:43:19.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f2524e0e700 1 Processor -- start 2024-04-25T11:43:19.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f2524e0e700 1 -- start start 2024-04-25T11:43:19.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f2524e0e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2520102ac0 0x7f2520198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f2524e0e700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2520103cc0 0x7f25201990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f2524e0e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2520104660 0x7f25201a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f2524e0e700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f252010e560 con 0x7f2520103cc0 2024-04-25T11:43:19.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f2524e0e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f252010e3e0 con 0x7f2520102ac0 2024-04-25T11:43:19.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f2524e0e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f252010e6e0 con 0x7f2520104660 2024-04-25T11:43:19.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f251dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2520103cc0 0x7f25201990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f251dd9b700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2520103cc0 0x7f25201990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54692/0 (socket says 172.21.15.83:54692) 2024-04-25T11:43:19.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f251dd9b700 1 -- 172.21.15.83:0/1754065094 learned_addr learned my addr 172.21.15.83:0/1754065094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:19.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f251ed9d700 1 --2- 172.21.15.83:0/1754065094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2520104660 0x7f25201a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f251e59c700 1 --2- 172.21.15.83:0/1754065094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2520102ac0 0x7f2520198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f251dd9b700 1 -- 172.21.15.83:0/1754065094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2520104660 msgr2=0x7f25201a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f251dd9b700 1 --2- 172.21.15.83:0/1754065094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2520104660 0x7f25201a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f251dd9b700 1 -- 172.21.15.83:0/1754065094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2520102ac0 msgr2=0x7f2520198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f251dd9b700 1 --2- 172.21.15.83:0/1754065094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2520102ac0 0x7f2520198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f251dd9b700 1 -- 172.21.15.83:0/1754065094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25201a0800 con 0x7f2520103cc0 2024-04-25T11:43:19.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.161+0000 7f251ed9d700 1 --2- 172.21.15.83:0/1754065094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2520104660 0x7f25201a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:19.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.162+0000 7f251dd9b700 1 --2- 172.21.15.83:0/1754065094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2520103cc0 0x7f25201990e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f251000c950 tx=0x7f251000cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:19.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.162+0000 7f250f7fe700 1 -- 172.21.15.83:0/1754065094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510007ac0 con 0x7f2520103cc0 2024-04-25T11:43:19.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.162+0000 7f2524e0e700 1 -- 172.21.15.83:0/1754065094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25201a0af0 con 0x7f2520103cc0 2024-04-25T11:43:19.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.162+0000 7f250f7fe700 1 -- 172.21.15.83:0/1754065094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2510004ce0 con 0x7f2520103cc0 2024-04-25T11:43:19.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.162+0000 7f250f7fe700 1 -- 172.21.15.83:0/1754065094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510005550 con 0x7f2520103cc0 2024-04-25T11:43:19.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.162+0000 7f2524e0e700 1 -- 172.21.15.83:0/1754065094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25201a1080 con 0x7f2520103cc0 2024-04-25T11:43:19.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.163+0000 7f2524e0e700 1 -- 172.21.15.83:0/1754065094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2520069730 con 0x7f2520103cc0 2024-04-25T11:43:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.164+0000 7f250f7fe700 1 -- 172.21.15.83:0/1754065094 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2510007c60 con 0x7f2520103cc0 2024-04-25T11:43:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.164+0000 7f250f7fe700 1 --2- 172.21.15.83:0/1754065094 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f25080a3cd0 0x7f25080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.164+0000 7f251e59c700 1 --2- 172.21.15.83:0/1754065094 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f25080a3cd0 0x7f25080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.164+0000 7f250f7fe700 1 -- 172.21.15.83:0/1754065094 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f25100d01d0 con 0x7f2520103cc0 2024-04-25T11:43:19.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.165+0000 7f251e59c700 1 --2- 172.21.15.83:0/1754065094 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f25080a3cd0 0x7f25080a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2514005950 tx=0x7f25140058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:19.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.169+0000 7f250f7fe700 1 -- 172.21.15.83:0/1754065094 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2510014030 con 0x7f2520103cc0 2024-04-25T11:43:19.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:19 smithi094 ceph-mon[33032]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:43:19.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:19 smithi111 ceph-mon[32784]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:43:19.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.374+0000 7f2524e0e700 1 -- 172.21.15.83:0/1754065094 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f2520062e80 con 0x7f25080a3cd0 2024-04-25T11:43:19.361 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:19 smithi083 ceph-mon[29330]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:43:19.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.376+0000 7f250f7fe700 1 -- 172.21.15.83:0/1754065094 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f2520062e80 con 0x7f25080a3cd0 2024-04-25T11:43:19.362 INFO:teuthology.orchestra.run.smithi083.stdout:HOST ADDR LABELS STATUS 2024-04-25T11:43:19.362 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083 172.21.15.83 2024-04-25T11:43:19.362 INFO:teuthology.orchestra.run.smithi083.stdout:smithi094 172.21.15.94 2024-04-25T11:43:19.362 INFO:teuthology.orchestra.run.smithi083.stdout:smithi111 172.21.15.111 2024-04-25T11:43:19.362 INFO:teuthology.orchestra.run.smithi083.stdout:3 hosts in cluster 2024-04-25T11:43:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.378+0000 7f2524e0e700 1 -- 172.21.15.83:0/1754065094 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f25080a3cd0 msgr2=0x7f25080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.378+0000 7f2524e0e700 1 --2- 172.21.15.83:0/1754065094 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f25080a3cd0 0x7f25080a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2514005950 tx=0x7f25140058e0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.378+0000 7f2524e0e700 1 -- 172.21.15.83:0/1754065094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2520103cc0 msgr2=0x7f25201990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.378+0000 7f2524e0e700 1 --2- 172.21.15.83:0/1754065094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2520103cc0 0x7f25201990e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f251000c950 tx=0x7f251000cd10 comp rx=0 tx=0).stop 2024-04-25T11:43:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.378+0000 7f2524e0e700 1 -- 172.21.15.83:0/1754065094 shutdown_connections 2024-04-25T11:43:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.378+0000 7f2524e0e700 1 --2- 172.21.15.83:0/1754065094 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f25080a3cd0 0x7f25080a6190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.378+0000 7f2524e0e700 1 --2- 172.21.15.83:0/1754065094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2520104660 0x7f25201a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.378+0000 7f2524e0e700 1 --2- 172.21.15.83:0/1754065094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2520102ac0 0x7f2520198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.378+0000 7f2524e0e700 1 --2- 172.21.15.83:0/1754065094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2520103cc0 0x7f25201990e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.378+0000 7f2524e0e700 1 -- 172.21.15.83:0/1754065094 >> 172.21.15.83:0/1754065094 conn(0x7f25200fe060 msgr2=0x7f2520107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.378+0000 7f2524e0e700 1 -- 172.21.15.83:0/1754065094 shutdown_connections 2024-04-25T11:43:19.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:19.378+0000 7f2524e0e700 1 -- 172.21.15.83:0/1754065094 wait complete. 2024-04-25T11:43:19.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T11:43:20.216 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:43:21.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:21 smithi094 ceph-mon[33032]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:21.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:21 smithi094 ceph-mon[33032]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T11:43:21.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:21 smithi094 ceph-mon[33032]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T11:43:21.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:21 smithi083 ceph-mon[29330]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:21.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:21 smithi083 ceph-mon[29330]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T11:43:21.289 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:21 smithi083 ceph-mon[29330]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T11:43:21.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:21 smithi111 ceph-mon[32784]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:21.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:21 smithi111 ceph-mon[32784]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-25T11:43:21.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:21 smithi111 ceph-mon[32784]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T11:43:21.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.809+0000 7f7d22386700 1 -- 172.21.15.83:0/4148563959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d1c102600 msgr2=0x7f7d1c109ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.809+0000 7f7d22386700 1 --2- 172.21.15.83:0/4148563959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d1c102600 0x7f7d1c109ec0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400b0d0 tx=0x7f7d1400b3e0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.809+0000 7f7d22386700 1 -- 172.21.15.83:0/4148563959 shutdown_connections 2024-04-25T11:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.809+0000 7f7d22386700 1 --2- 172.21.15.83:0/4148563959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d1c102600 0x7f7d1c109ec0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.809+0000 7f7d22386700 1 --2- 172.21.15.83:0/4148563959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d1c102f20 0x7f7d1c103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.809+0000 7f7d22386700 1 --2- 172.21.15.83:0/4148563959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d1c101c60 0x7f7d1c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.809+0000 7f7d22386700 1 -- 172.21.15.83:0/4148563959 >> 172.21.15.83:0/4148563959 conn(0x7f7d1c0fc6d0 msgr2=0x7f7d1c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.809+0000 7f7d22386700 1 -- 172.21.15.83:0/4148563959 shutdown_connections 2024-04-25T11:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.809+0000 7f7d22386700 1 -- 172.21.15.83:0/4148563959 wait complete. 2024-04-25T11:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.809+0000 7f7d22386700 1 Processor -- start 2024-04-25T11:43:21.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.809+0000 7f7d22386700 1 -- start start 2024-04-25T11:43:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d22386700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d1c101c60 0x7f7d1c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d22386700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d1c102600 0x7f7d1c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d22386700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d1c102f20 0x7f7d1c1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d22386700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c10d570 con 0x7f7d1c102600 2024-04-25T11:43:21.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d22386700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c10d3f0 con 0x7f7d1c101c60 2024-04-25T11:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d22386700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c10d6f0 con 0x7f7d1c102f20 2024-04-25T11:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d1b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d1c102600 0x7f7d1c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d1b7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d1c102600 0x7f7d1c199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54714/0 (socket says 172.21.15.83:54714) 2024-04-25T11:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d1b7fe700 1 -- 172.21.15.83:0/3334650804 learned_addr learned my addr 172.21.15.83:0/3334650804 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d20923700 1 --2- 172.21.15.83:0/3334650804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d1c102f20 0x7f7d1c1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d1bfff700 1 --2- 172.21.15.83:0/3334650804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d1c101c60 0x7f7d1c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d1b7fe700 1 -- 172.21.15.83:0/3334650804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d1c102f20 msgr2=0x7f7d1c1a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d1b7fe700 1 --2- 172.21.15.83:0/3334650804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d1c102f20 0x7f7d1c1a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d1b7fe700 1 -- 172.21.15.83:0/3334650804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d1c101c60 msgr2=0x7f7d1c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d1b7fe700 1 --2- 172.21.15.83:0/3334650804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d1c101c60 0x7f7d1c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d1b7fe700 1 -- 172.21.15.83:0/3334650804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c1a0990 con 0x7f7d1c102600 2024-04-25T11:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d1bfff700 1 --2- 172.21.15.83:0/3334650804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d1c101c60 0x7f7d1c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.810+0000 7f7d1b7fe700 1 --2- 172.21.15.83:0/3334650804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d1c102600 0x7f7d1c199270 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000b560 tx=0x7f7d1000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.811+0000 7f7d197fa700 1 -- 172.21.15.83:0/3334650804 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10015020 con 0x7f7d1c102600 2024-04-25T11:43:21.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.811+0000 7f7d22386700 1 -- 172.21.15.83:0/3334650804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1c1a0c80 con 0x7f7d1c102600 2024-04-25T11:43:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.811+0000 7f7d197fa700 1 -- 172.21.15.83:0/3334650804 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d10011ce0 con 0x7f7d1c102600 2024-04-25T11:43:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.811+0000 7f7d22386700 1 -- 172.21.15.83:0/3334650804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d1c1a1240 con 0x7f7d1c102600 2024-04-25T11:43:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.811+0000 7f7d197fa700 1 -- 172.21.15.83:0/3334650804 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10010d20 con 0x7f7d1c102600 2024-04-25T11:43:21.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.811+0000 7f7d22386700 1 -- 172.21.15.83:0/3334650804 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d1c069730 con 0x7f7d1c102600 2024-04-25T11:43:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.813+0000 7f7d197fa700 1 -- 172.21.15.83:0/3334650804 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7d10018020 con 0x7f7d1c102600 2024-04-25T11:43:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.813+0000 7f7d197fa700 1 --2- 172.21.15.83:0/3334650804 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7d040a3cc0 0x7f7d040a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.813+0000 7f7d1bfff700 1 --2- 172.21.15.83:0/3334650804 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7d040a3cc0 0x7f7d040a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.813+0000 7f7d197fa700 1 -- 172.21.15.83:0/3334650804 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f7d100cffb0 con 0x7f7d1c102600 2024-04-25T11:43:21.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.814+0000 7f7d1bfff700 1 --2- 172.21.15.83:0/3334650804 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7d040a3cc0 0x7f7d040a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c06a4a0 tx=0x7f7d0c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:21.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:21.817+0000 7f7d197fa700 1 -- 172.21.15.83:0/3334650804 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d1001f020 con 0x7f7d1c102600 2024-04-25T11:43:22.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:22.026+0000 7f7d22386700 1 -- 172.21.15.83:0/3334650804 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f7d1c0fe900 con 0x7f7d040a3cc0 2024-04-25T11:43:22.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:22.028+0000 7f7d197fa700 1 -- 172.21.15.83:0/3334650804 <== mgr.14152 v2:172.21.15.83:6800/1157946519 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f7d1c0fe900 con 0x7f7d040a3cc0 2024-04-25T11:43:22.014 INFO:teuthology.orchestra.run.smithi083.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T11:43:22.015 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040148400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T11:43:22.015 INFO:teuthology.orchestra.run.smithi083.stdout:smithi094 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001L480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-04-25T11:43:22.015 INFO:teuthology.orchestra.run.smithi083.stdout:smithi111 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VZ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T11:43:22.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:22.030+0000 7f7d22386700 1 -- 172.21.15.83:0/3334650804 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7d040a3cc0 msgr2=0x7f7d040a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:22.030+0000 7f7d22386700 1 --2- 172.21.15.83:0/3334650804 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7d040a3cc0 0x7f7d040a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c06a4a0 tx=0x7f7d0c008040 comp rx=0 tx=0).stop 2024-04-25T11:43:22.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:22.030+0000 7f7d22386700 1 -- 172.21.15.83:0/3334650804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d1c102600 msgr2=0x7f7d1c199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:22.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:22.030+0000 7f7d22386700 1 --2- 172.21.15.83:0/3334650804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d1c102600 0x7f7d1c199270 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000b560 tx=0x7f7d1000b870 comp rx=0 tx=0).stop 2024-04-25T11:43:22.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:22.030+0000 7f7d22386700 1 -- 172.21.15.83:0/3334650804 shutdown_connections 2024-04-25T11:43:22.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:22.030+0000 7f7d22386700 1 --2- 172.21.15.83:0/3334650804 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f7d040a3cc0 0x7f7d040a6180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:22.030+0000 7f7d22386700 1 --2- 172.21.15.83:0/3334650804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d1c102f20 0x7f7d1c1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:22.030+0000 7f7d22386700 1 --2- 172.21.15.83:0/3334650804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7d1c101c60 0x7f7d1c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:22.030+0000 7f7d22386700 1 --2- 172.21.15.83:0/3334650804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d1c102600 0x7f7d1c199270 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:22.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:22.030+0000 7f7d22386700 1 -- 172.21.15.83:0/3334650804 >> 172.21.15.83:0/3334650804 conn(0x7f7d1c0fc6d0 msgr2=0x7f7d1c0fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:22.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:22.030+0000 7f7d22386700 1 -- 172.21.15.83:0/3334650804 shutdown_connections 2024-04-25T11:43:22.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:22.030+0000 7f7d22386700 1 -- 172.21.15.83:0/3334650804 wait complete. 2024-04-25T11:43:22.654 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T11:43:22.663 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T11:43:22.663 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T11:43:22.907 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:43:23.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:23 smithi094 ceph-mon[33032]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-25T11:43:23.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:23 smithi094 ceph-mon[33032]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:23.274 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:23 smithi083 ceph-mon[29330]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-25T11:43:23.274 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:23 smithi083 ceph-mon[29330]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:23.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:23 smithi111 ceph-mon[32784]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-25T11:43:23.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:23 smithi111 ceph-mon[32784]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.537+0000 7ff5b3d82700 1 -- 172.21.15.83:0/1056020987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac102aa0 msgr2=0x7ff5ac102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.537+0000 7ff5b3d82700 1 --2- 172.21.15.83:0/1056020987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac102aa0 0x7ff5ac102ec0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80099c0 tx=0x7ff5a8009cd0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.537+0000 7ff5b3d82700 1 -- 172.21.15.83:0/1056020987 shutdown_connections 2024-04-25T11:43:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.537+0000 7ff5b3d82700 1 --2- 172.21.15.83:0/1056020987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ac104640 0x7ff5ac10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.537+0000 7ff5b3d82700 1 --2- 172.21.15.83:0/1056020987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac102aa0 0x7ff5ac102ec0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.537+0000 7ff5b3d82700 1 --2- 172.21.15.83:0/1056020987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac103ca0 0x7ff5ac104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.537+0000 7ff5b3d82700 1 -- 172.21.15.83:0/1056020987 >> 172.21.15.83:0/1056020987 conn(0x7ff5ac0fe040 msgr2=0x7ff5ac100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.537+0000 7ff5b3d82700 1 -- 172.21.15.83:0/1056020987 shutdown_connections 2024-04-25T11:43:24.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.537+0000 7ff5b3d82700 1 -- 172.21.15.83:0/1056020987 wait complete. 2024-04-25T11:43:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.538+0000 7ff5b3d82700 1 Processor -- start 2024-04-25T11:43:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.538+0000 7ff5b3d82700 1 -- start start 2024-04-25T11:43:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.538+0000 7ff5b3d82700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ac102aa0 0x7ff5ac198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.538+0000 7ff5b3d82700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103ca0 0x7ff5ac199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.538+0000 7ff5b3d82700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac104640 0x7ff5ac1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.538+0000 7ff5b3d82700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac10e590 con 0x7ff5ac103ca0 2024-04-25T11:43:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.538+0000 7ff5b3d82700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac10e410 con 0x7ff5ac104640 2024-04-25T11:43:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.538+0000 7ff5b3d82700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac10e710 con 0x7ff5ac102aa0 2024-04-25T11:43:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.538+0000 7ff5b131d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103ca0 0x7ff5ac199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.538+0000 7ff5b131d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103ca0 0x7ff5ac199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48456/0 (socket says 172.21.15.83:48456) 2024-04-25T11:43:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.538+0000 7ff5b131d700 1 -- 172.21.15.83:0/1322079736 learned_addr learned my addr 172.21.15.83:0/1322079736 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.539+0000 7ff5b231f700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac104640 0x7ff5ac1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.539+0000 7ff5b1b1e700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ac102aa0 0x7ff5ac198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.539+0000 7ff5b131d700 1 -- 172.21.15.83:0/1322079736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ac102aa0 msgr2=0x7ff5ac198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.539+0000 7ff5b131d700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ac102aa0 0x7ff5ac198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.539+0000 7ff5b131d700 1 -- 172.21.15.83:0/1322079736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac104640 msgr2=0x7ff5ac1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.539+0000 7ff5b131d700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac104640 0x7ff5ac1a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.539+0000 7ff5b131d700 1 -- 172.21.15.83:0/1322079736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ac1a0870 con 0x7ff5ac103ca0 2024-04-25T11:43:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.539+0000 7ff5b1b1e700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ac102aa0 0x7ff5ac198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.539+0000 7ff5b231f700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac104640 0x7ff5ac1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.539+0000 7ff5b131d700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103ca0 0x7ff5ac199160 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00c9b0 tx=0x7ff59c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.539+0000 7ff5a2ffd700 1 -- 172.21.15.83:0/1322079736 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c007b00 con 0x7ff5ac103ca0 2024-04-25T11:43:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.539+0000 7ff5b3d82700 1 -- 172.21.15.83:0/1322079736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ac1a0b60 con 0x7ff5ac103ca0 2024-04-25T11:43:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.539+0000 7ff5a2ffd700 1 -- 172.21.15.83:0/1322079736 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff59c007ca0 con 0x7ff5ac103ca0 2024-04-25T11:43:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.539+0000 7ff5a2ffd700 1 -- 172.21.15.83:0/1322079736 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c018730 con 0x7ff5ac103ca0 2024-04-25T11:43:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.539+0000 7ff5b3d82700 1 -- 172.21.15.83:0/1322079736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ac1a10f0 con 0x7ff5ac103ca0 2024-04-25T11:43:24.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.540+0000 7ff5b3d82700 1 -- 172.21.15.83:0/1322079736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5ac02df80 con 0x7ff5ac103ca0 2024-04-25T11:43:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.541+0000 7ff5a2ffd700 1 -- 172.21.15.83:0/1322079736 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff59c00f450 con 0x7ff5ac103ca0 2024-04-25T11:43:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.542+0000 7ff5a2ffd700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff5980a3c40 0x7ff5980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.542+0000 7ff5b1b1e700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff5980a3c40 0x7ff5980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.542+0000 7ff5a2ffd700 1 -- 172.21.15.83:0/1322079736 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7ff59c0d0480 con 0x7ff5ac103ca0 2024-04-25T11:43:24.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.542+0000 7ff5b1b1e700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff5980a3c40 0x7ff5980a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8000c00 tx=0x7ff5a8009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:24.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.546+0000 7ff5a2ffd700 1 -- 172.21.15.83:0/1322079736 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff59c014030 con 0x7ff5ac103ca0 2024-04-25T11:43:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.793+0000 7ff5b3d82700 1 -- 172.21.15.83:0/1322079736 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff5ac063650 con 0x7ff5ac103ca0 2024-04-25T11:43:24.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.793+0000 7ff5a2ffd700 1 -- 172.21.15.83:0/1322079736 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3090 (secure 0 0 0) 0x7ff59c09ce70 con 0x7ff5ac103ca0 2024-04-25T11:43:24.779 INFO:teuthology.orchestra.run.smithi083.stdout:e24 2024-04-25T11:43:24.779 INFO:teuthology.orchestra.run.smithi083.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T11:43:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T11:43:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:legacy client fscid: 1 2024-04-25T11:43:24.780 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:Filesystem 'cephfs' (1) 2024-04-25T11:43:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:fs_name cephfs 2024-04-25T11:43:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 24 2024-04-25T11:43:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T11:43:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-04-25T11:41:44.787926+0000 2024-04-25T11:43:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:modified 2024-04-25T11:41:51.578220+0000 2024-04-25T11:43:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:tableserver 0 2024-04-25T11:43:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:root 0 2024-04-25T11:43:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:session_timeout 300 2024-04-25T11:43:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:session_autoclose 300 2024-04-25T11:43:24.780 INFO:teuthology.orchestra.run.smithi083.stdout:max_file_size 1099511627776 2024-04-25T11:43:24.781 INFO:teuthology.orchestra.run.smithi083.stdout:max_xattr_size 65536 2024-04-25T11:43:24.781 INFO:teuthology.orchestra.run.smithi083.stdout:required_client_features {} 2024-04-25T11:43:24.781 INFO:teuthology.orchestra.run.smithi083.stdout:last_failure 0 2024-04-25T11:43:24.781 INFO:teuthology.orchestra.run.smithi083.stdout:last_failure_osd_epoch 0 2024-04-25T11:43:24.781 INFO:teuthology.orchestra.run.smithi083.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T11:43:24.781 INFO:teuthology.orchestra.run.smithi083.stdout:max_mds 5 2024-04-25T11:43:24.781 INFO:teuthology.orchestra.run.smithi083.stdout:in 0,1,2,3,4 2024-04-25T11:43:24.781 INFO:teuthology.orchestra.run.smithi083.stdout:up {0=24479,1=24469,2=24442,3=24440,4=24430} 2024-04-25T11:43:24.781 INFO:teuthology.orchestra.run.smithi083.stdout:failed 2024-04-25T11:43:24.781 INFO:teuthology.orchestra.run.smithi083.stdout:damaged 2024-04-25T11:43:24.781 INFO:teuthology.orchestra.run.smithi083.stdout:stopped 2024-04-25T11:43:24.781 INFO:teuthology.orchestra.run.smithi083.stdout:data_pools [3] 2024-04-25T11:43:24.781 INFO:teuthology.orchestra.run.smithi083.stdout:metadata_pool 2 2024-04-25T11:43:24.781 INFO:teuthology.orchestra.run.smithi083.stdout:inline_data disabled 2024-04-25T11:43:24.782 INFO:teuthology.orchestra.run.smithi083.stdout:balancer 2024-04-25T11:43:24.782 INFO:teuthology.orchestra.run.smithi083.stdout:bal_rank_mask -1 2024-04-25T11:43:24.782 INFO:teuthology.orchestra.run.smithi083.stdout:standby_count_wanted 0 2024-04-25T11:43:24.782 INFO:teuthology.orchestra.run.smithi083.stdout:qdb_cluster leader: 24479 members: 24440,24430,24479,24469,24442 2024-04-25T11:43:24.782 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b{0:24479} state up:active seq 3 addr [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:43:24.782 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.e{0:24416} state up:standby-replay seq 1 addr [v2:172.21.15.94:6834/4086957377,v1:172.21.15.94:6835/4086957377] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:43:24.782 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.i{1:24469} state up:active seq 5 addr [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:43:24.782 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.k{1:24392} state up:standby-replay seq 1 addr [v2:172.21.15.94:6832/3250237836,v1:172.21.15.94:6833/3250237836] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:43:24.782 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c{2:24442} state up:active seq 7 addr [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:43:24.782 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.l{2:24391} state up:standby-replay seq 1 addr [v2:172.21.15.111:6832/2234401967,v1:172.21.15.111:6833/2234401967] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:43:24.782 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.h{3:24440} state up:active seq 7 addr [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:43:24.782 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a{3:14643} state up:standby-replay seq 1 addr [v2:172.21.15.83:6840/1576544313,v1:172.21.15.83:6841/1576544313] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:43:24.782 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.f{4:24430} state up:active seq 8 addr [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:43:24.782 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.g{4:14631} state up:standby-replay seq 1 addr [v2:172.21.15.83:6838/1568064835,v1:172.21.15.83:6839/1568064835] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:43:24.783 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:24.783 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:24.783 INFO:teuthology.orchestra.run.smithi083.stdout:Standby daemons: 2024-04-25T11:43:24.783 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:24.783 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.j{-1:14562} state up:standby seq 1 addr [v2:172.21.15.83:6834/3911246904,v1:172.21.15.83:6835/3911246904] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:43:24.783 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d{-1:14574} state up:standby seq 1 addr [v2:172.21.15.83:6836/3006959432,v1:172.21.15.83:6837/3006959432] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T11:43:24.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.795+0000 7ff5b3d82700 1 -- 172.21.15.83:0/1322079736 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff5980a3c40 msgr2=0x7ff5980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.795+0000 7ff5b3d82700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff5980a3c40 0x7ff5980a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8000c00 tx=0x7ff5a8009f10 comp rx=0 tx=0).stop 2024-04-25T11:43:24.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.795+0000 7ff5b3d82700 1 -- 172.21.15.83:0/1322079736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103ca0 msgr2=0x7ff5ac199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:24.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.795+0000 7ff5b3d82700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103ca0 0x7ff5ac199160 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00c9b0 tx=0x7ff59c00cd70 comp rx=0 tx=0).stop 2024-04-25T11:43:24.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.795+0000 7ff5b3d82700 1 -- 172.21.15.83:0/1322079736 shutdown_connections 2024-04-25T11:43:24.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.795+0000 7ff5b3d82700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff5980a3c40 0x7ff5980a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.795+0000 7ff5b3d82700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5ac104640 0x7ff5ac1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.795+0000 7ff5b3d82700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5ac102aa0 0x7ff5ac198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.795+0000 7ff5b3d82700 1 --2- 172.21.15.83:0/1322079736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff5ac103ca0 0x7ff5ac199160 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:24.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.795+0000 7ff5b3d82700 1 -- 172.21.15.83:0/1322079736 >> 172.21.15.83:0/1322079736 conn(0x7ff5ac0fe040 msgr2=0x7ff5ac107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:24.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.795+0000 7ff5b3d82700 1 -- 172.21.15.83:0/1322079736 shutdown_connections 2024-04-25T11:43:24.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:24.795+0000 7ff5b3d82700 1 -- 172.21.15.83:0/1322079736 wait complete. 2024-04-25T11:43:24.784 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-04-25T11:43:25.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:25 smithi083 ceph-mon[29330]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T11:43:25.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:25 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1322079736' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T11:43:25.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:25 smithi111 ceph-mon[32784]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T11:43:25.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:25 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1322079736' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T11:43:25.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:25 smithi094 ceph-mon[33032]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-25T11:43:25.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:25 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1322079736' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T11:43:25.591 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T11:43:25.830 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:43:27.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:27 smithi083 ceph-mon[29330]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T11:43:27.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:27 smithi111 ceph-mon[32784]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T11:43:27.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.454+0000 7f3c94263700 1 -- 172.21.15.83:0/2589035070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8c105ef0 msgr2=0x7f3c8c1082e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:27.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.454+0000 7f3c94263700 1 --2- 172.21.15.83:0/2589035070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8c105ef0 0x7f3c8c1082e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009990 tx=0x7f3c7c009ca0 comp rx=0 tx=0).stop 2024-04-25T11:43:27.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.454+0000 7f3c94263700 1 -- 172.21.15.83:0/2589035070 shutdown_connections 2024-04-25T11:43:27.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.454+0000 7f3c94263700 1 --2- 172.21.15.83:0/2589035070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c8c108820 0x7f3c8c10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.454+0000 7f3c94263700 1 --2- 172.21.15.83:0/2589035070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c8c1035c0 0x7f3c8c1059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.454+0000 7f3c94263700 1 --2- 172.21.15.83:0/2589035070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8c105ef0 0x7f3c8c1082e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.454+0000 7f3c94263700 1 -- 172.21.15.83:0/2589035070 >> 172.21.15.83:0/2589035070 conn(0x7f3c8c0fcf70 msgr2=0x7f3c8c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.454+0000 7f3c94263700 1 -- 172.21.15.83:0/2589035070 shutdown_connections 2024-04-25T11:43:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.454+0000 7f3c94263700 1 -- 172.21.15.83:0/2589035070 wait complete. 2024-04-25T11:43:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c94263700 1 Processor -- start 2024-04-25T11:43:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c94263700 1 -- start start 2024-04-25T11:43:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c94263700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c8c1035c0 0x7f3c8c198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c94263700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c8c105ef0 0x7f3c8c1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c94263700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8c108820 0x7f3c8c1a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c94263700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c10e5b0 con 0x7f3c8c108820 2024-04-25T11:43:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c94263700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c10e430 con 0x7f3c8c1035c0 2024-04-25T11:43:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c94263700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c10e730 con 0x7f3c8c105ef0 2024-04-25T11:43:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c92800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8c108820 0x7f3c8c1a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:27.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c92800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8c108820 0x7f3c8c1a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48478/0 (socket says 172.21.15.83:48478) 2024-04-25T11:43:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c92800700 1 -- 172.21.15.83:0/3571514197 learned_addr learned my addr 172.21.15.83:0/3571514197 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c917fe700 1 --2- 172.21.15.83:0/3571514197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c8c105ef0 0x7f3c8c1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c91fff700 1 --2- 172.21.15.83:0/3571514197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c8c1035c0 0x7f3c8c198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c92800700 1 -- 172.21.15.83:0/3571514197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c8c105ef0 msgr2=0x7f3c8c1990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c92800700 1 --2- 172.21.15.83:0/3571514197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c8c105ef0 0x7f3c8c1990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c92800700 1 -- 172.21.15.83:0/3571514197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c8c1035c0 msgr2=0x7f3c8c198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c92800700 1 --2- 172.21.15.83:0/3571514197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c8c1035c0 0x7f3c8c198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c92800700 1 -- 172.21.15.83:0/3571514197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8c1a07f0 con 0x7f3c8c108820 2024-04-25T11:43:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.455+0000 7f3c91fff700 1 --2- 172.21.15.83:0/3571514197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c8c1035c0 0x7f3c8c198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.456+0000 7f3c92800700 1 --2- 172.21.15.83:0/3571514197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8c108820 0x7f3c8c1a0150 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400ba10 tx=0x7f3c8400bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.456+0000 7f3c82ffd700 1 -- 172.21.15.83:0/3571514197 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84019070 con 0x7f3c8c108820 2024-04-25T11:43:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.456+0000 7f3c94263700 1 -- 172.21.15.83:0/3571514197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c8c1a0ae0 con 0x7f3c8c108820 2024-04-25T11:43:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.456+0000 7f3c82ffd700 1 -- 172.21.15.83:0/3571514197 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c8400cde0 con 0x7f3c8c108820 2024-04-25T11:43:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.456+0000 7f3c82ffd700 1 -- 172.21.15.83:0/3571514197 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c84012340 con 0x7f3c8c108820 2024-04-25T11:43:27.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.456+0000 7f3c94263700 1 -- 172.21.15.83:0/3571514197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c8c1a0fd0 con 0x7f3c8c108820 2024-04-25T11:43:27.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.457+0000 7f3c94263700 1 -- 172.21.15.83:0/3571514197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c8c069730 con 0x7f3c8c108820 2024-04-25T11:43:27.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.458+0000 7f3c82ffd700 1 -- 172.21.15.83:0/3571514197 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3c8401c070 con 0x7f3c8c108820 2024-04-25T11:43:27.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.459+0000 7f3c82ffd700 1 --2- 172.21.15.83:0/3571514197 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f3c780a3c40 0x7f3c780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:27.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.459+0000 7f3c91fff700 1 --2- 172.21.15.83:0/3571514197 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f3c780a3c40 0x7f3c780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:27.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.459+0000 7f3c82ffd700 1 -- 172.21.15.83:0/3571514197 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f3c840cf840 con 0x7f3c8c108820 2024-04-25T11:43:27.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.459+0000 7f3c91fff700 1 --2- 172.21.15.83:0/3571514197 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f3c780a3c40 0x7f3c780a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3c88005fd0 tx=0x7f3c88005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:27.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.463+0000 7f3c82ffd700 1 -- 172.21.15.83:0/3571514197 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c8409c230 con 0x7f3c8c108820 2024-04-25T11:43:27.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:27 smithi094 ceph-mon[33032]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-25T11:43:27.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.664+0000 7f3c94263700 1 -- 172.21.15.83:0/3571514197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f3c8c063650 con 0x7f3c8c108820 2024-04-25T11:43:27.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.665+0000 7f3c82ffd700 1 -- 172.21.15.83:0/3571514197 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5260 (secure 0 0 0) 0x7f3c8409bb30 con 0x7f3c8c108820 2024-04-25T11:43:27.651 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 88 2024-04-25T11:43:27.651 INFO:teuthology.orchestra.run.smithi083.stdout:fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:43:27.651 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-04-25T11:33:56.406187+0000 2024-04-25T11:43:27.652 INFO:teuthology.orchestra.run.smithi083.stdout:modified 2024-04-25T11:42:51.074495+0000 2024-04-25T11:43:27.652 INFO:teuthology.orchestra.run.smithi083.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T11:43:27.652 INFO:teuthology.orchestra.run.smithi083.stdout:crush_version 26 2024-04-25T11:43:27.652 INFO:teuthology.orchestra.run.smithi083.stdout:full_ratio 0.95 2024-04-25T11:43:27.652 INFO:teuthology.orchestra.run.smithi083.stdout:backfillfull_ratio 0.9 2024-04-25T11:43:27.652 INFO:teuthology.orchestra.run.smithi083.stdout:nearfull_ratio 0.85 2024-04-25T11:43:27.652 INFO:teuthology.orchestra.run.smithi083.stdout:require_min_compat_client luminous 2024-04-25T11:43:27.652 INFO:teuthology.orchestra.run.smithi083.stdout:min_compat_client luminous 2024-04-25T11:43:27.652 INFO:teuthology.orchestra.run.smithi083.stdout:require_osd_release squid 2024-04-25T11:43:27.652 INFO:teuthology.orchestra.run.smithi083.stdout:stretch_mode_enabled false 2024-04-25T11:43:27.652 INFO:teuthology.orchestra.run.smithi083.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T11:43:27.652 INFO:teuthology.orchestra.run.smithi083.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T11:43:27.652 INFO:teuthology.orchestra.run.smithi083.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T11:43:27.652 INFO:teuthology.orchestra.run.smithi083.stdout:max_osd 12 2024-04-25T11:43:27.653 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6802/2168030106,v1:172.21.15.83:6803/2168030106] [v2:172.21.15.83:6804/2168030106,v1:172.21.15.83:6805/2168030106] exists,up a79d318f-21ba-4f1b-a80c-54a8fa4618b0 2024-04-25T11:43:27.653 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6800/3582270380,v1:172.21.15.94:6801/3582270380] [v2:172.21.15.94:6802/3582270380,v1:172.21.15.94:6803/3582270380] exists,up e4540152-b87e-49a9-bf80-6de7cb165835 2024-04-25T11:43:27.653 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6800/101429764,v1:172.21.15.111:6801/101429764] [v2:172.21.15.111:6802/101429764,v1:172.21.15.111:6803/101429764] exists,up bdc6f455-7038-4178-b962-2de2d69180cb 2024-04-25T11:43:27.653 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6810/775479209,v1:172.21.15.83:6811/775479209] [v2:172.21.15.83:6812/775479209,v1:172.21.15.83:6813/775479209] exists,up 1e927e50-a208-48ca-aeb4-1adc2ae48d55 2024-04-25T11:43:27.653 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6808/768239394,v1:172.21.15.94:6809/768239394] [v2:172.21.15.94:6810/768239394,v1:172.21.15.94:6811/768239394] exists,up 637e5b1b-12cb-41e9-9026-53faadfc28b6 2024-04-25T11:43:27.653 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6808/3244351490,v1:172.21.15.111:6809/3244351490] [v2:172.21.15.111:6810/3244351490,v1:172.21.15.111:6811/3244351490] exists,up 050ab73c-e5d2-4140-8118-8f7c65b40229 2024-04-25T11:43:27.653 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6818/305831773,v1:172.21.15.83:6819/305831773] [v2:172.21.15.83:6820/305831773,v1:172.21.15.83:6821/305831773] exists,up 44e6b825-a33f-447c-bc4c-f1890016bd3e 2024-04-25T11:43:27.653 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7 up in weight 1 up_from 52 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6816/4200260463,v1:172.21.15.94:6817/4200260463] [v2:172.21.15.94:6818/4200260463,v1:172.21.15.94:6819/4200260463] exists,up 50476b46-ec06-472f-bb91-920d81555099 2024-04-25T11:43:27.653 INFO:teuthology.orchestra.run.smithi083.stdout:osd.8 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6816/297826032,v1:172.21.15.111:6817/297826032] [v2:172.21.15.111:6818/297826032,v1:172.21.15.111:6819/297826032] exists,up 34ebcd68-bca8-4d62-b0df-728fd892a0b5 2024-04-25T11:43:27.653 INFO:teuthology.orchestra.run.smithi083.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6826/3768581100,v1:172.21.15.83:6827/3768581100] [v2:172.21.15.83:6828/3768581100,v1:172.21.15.83:6829/3768581100] exists,up 9f70803a-8d30-40cd-8374-fefeb1bfc337 2024-04-25T11:43:27.653 INFO:teuthology.orchestra.run.smithi083.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6824/2135498753,v1:172.21.15.94:6825/2135498753] [v2:172.21.15.94:6826/2135498753,v1:172.21.15.94:6827/2135498753] exists,up 7d3d3f69-db95-47f3-aced-3eaa340839da 2024-04-25T11:43:27.653 INFO:teuthology.orchestra.run.smithi083.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6824/4117294468,v1:172.21.15.111:6825/4117294468] [v2:172.21.15.111:6826/4117294468,v1:172.21.15.111:6827/4117294468] exists,up 16ab0143-02e4-4a08-8cd3-3c9d9c841ad5 2024-04-25T11:43:27.653 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.4 [2,11] 2024-04-25T11:43:27.653 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.1d [8,11] 2024-04-25T11:43:27.654 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.7 [3,9] 2024-04-25T11:43:27.654 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.11 [5,9] 2024-04-25T11:43:27.654 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.17 [4,6] 2024-04-25T11:43:27.654 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.1a [5,7] 2024-04-25T11:43:27.654 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.23 [4,8] 2024-04-25T11:43:27.654 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.2c [4,9] 2024-04-25T11:43:27.654 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/486441637 expires 2024-04-26T11:34:49.826109+0000 2024-04-25T11:43:27.654 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/1048355479 expires 2024-04-26T11:34:49.826109+0000 2024-04-25T11:43:27.654 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6801/1639787889 expires 2024-04-26T11:34:49.826109+0000 2024-04-25T11:43:27.654 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/1639787889 expires 2024-04-26T11:34:49.826109+0000 2024-04-25T11:43:27.654 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3257452011 expires 2024-04-26T11:34:23.164299+0000 2024-04-25T11:43:27.654 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/908199980 expires 2024-04-26T11:34:49.826109+0000 2024-04-25T11:43:27.655 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/3997740373 expires 2024-04-26T11:34:23.164299+0000 2024-04-25T11:43:27.655 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/2677315213 expires 2024-04-26T11:34:23.164299+0000 2024-04-25T11:43:27.655 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6801/3997740373 expires 2024-04-26T11:34:23.164299+0000 2024-04-25T11:43:27.655 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/1174617340 expires 2024-04-26T11:34:23.164299+0000 2024-04-25T11:43:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.667+0000 7f3c94263700 1 -- 172.21.15.83:0/3571514197 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f3c780a3c40 msgr2=0x7f3c780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.667+0000 7f3c94263700 1 --2- 172.21.15.83:0/3571514197 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f3c780a3c40 0x7f3c780a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3c88005fd0 tx=0x7f3c88005e30 comp rx=0 tx=0).stop 2024-04-25T11:43:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.667+0000 7f3c94263700 1 -- 172.21.15.83:0/3571514197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8c108820 msgr2=0x7f3c8c1a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.667+0000 7f3c94263700 1 --2- 172.21.15.83:0/3571514197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8c108820 0x7f3c8c1a0150 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400ba10 tx=0x7f3c8400bd20 comp rx=0 tx=0).stop 2024-04-25T11:43:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.667+0000 7f3c94263700 1 -- 172.21.15.83:0/3571514197 shutdown_connections 2024-04-25T11:43:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.667+0000 7f3c94263700 1 --2- 172.21.15.83:0/3571514197 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f3c780a3c40 0x7f3c780a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.667+0000 7f3c94263700 1 --2- 172.21.15.83:0/3571514197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c8c108820 0x7f3c8c1a0150 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.667+0000 7f3c94263700 1 --2- 172.21.15.83:0/3571514197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c8c1035c0 0x7f3c8c198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.667+0000 7f3c94263700 1 --2- 172.21.15.83:0/3571514197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c8c105ef0 0x7f3c8c1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.667+0000 7f3c94263700 1 -- 172.21.15.83:0/3571514197 >> 172.21.15.83:0/3571514197 conn(0x7f3c8c0fcf70 msgr2=0x7f3c8c103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:27.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.667+0000 7f3c94263700 1 -- 172.21.15.83:0/3571514197 shutdown_connections 2024-04-25T11:43:27.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:27.667+0000 7f3c94263700 1 -- 172.21.15.83:0/3571514197 wait complete. 2024-04-25T11:43:28.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:28 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3571514197' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T11:43:28.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:28 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3571514197' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T11:43:28.456 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T11:43:28.467 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:43:28.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:28 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3571514197' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T11:43:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.582+0000 7fe057c13640 1 -- 172.21.15.83:0/2830056533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0501053e0 msgr2=0x7fe0501077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.582+0000 7fe057c13640 1 --2- 172.21.15.83:0/2830056533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0501053e0 0x7fe0501077d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe040009900 tx=0x7fe04002a410 comp rx=0 tx=0).stop 2024-04-25T11:43:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.582+0000 7fe057c13640 1 -- 172.21.15.83:0/2830056533 shutdown_connections 2024-04-25T11:43:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.582+0000 7fe057c13640 1 --2- 172.21.15.83:0/2830056533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0501053e0 0x7fe0501077d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.582+0000 7fe057c13640 1 --2- 172.21.15.83:0/2830056533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe050107d10 0x7fe05010a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.582+0000 7fe057c13640 1 --2- 172.21.15.83:0/2830056533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe050102ab0 0x7fe050104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.582+0000 7fe057c13640 1 -- 172.21.15.83:0/2830056533 >> 172.21.15.83:0/2830056533 conn(0x7fe0500fc400 msgr2=0x7fe0500fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.582+0000 7fe057c13640 1 -- 172.21.15.83:0/2830056533 shutdown_connections 2024-04-25T11:43:28.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.582+0000 7fe057c13640 1 -- 172.21.15.83:0/2830056533 wait complete. 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.582+0000 7fe057c13640 1 Processor -- start 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.582+0000 7fe057c13640 1 -- start start 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe057c13640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe050102ab0 0x7fe050198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe057c13640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0501053e0 0x7fe050198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe057c13640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe050107d10 0x7fe05019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe057c13640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe05010db40 con 0x7fe050102ab0 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe057c13640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe05010d9c0 con 0x7fe050107d10 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe057c13640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe05010dcc0 con 0x7fe0501053e0 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe0559af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe050102ab0 0x7fe050198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe0559af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe050102ab0 0x7fe050198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48510/0 (socket says 172.21.15.83:48510) 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe0559af640 1 -- 172.21.15.83:0/2439466573 learned_addr learned my addr 172.21.15.83:0/2439466573 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe0561b0640 1 --2- 172.21.15.83:0/2439466573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe050107d10 0x7fe05019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe0551ae640 1 --2- 172.21.15.83:0/2439466573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0501053e0 0x7fe050198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe0559af640 1 -- 172.21.15.83:0/2439466573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0501053e0 msgr2=0x7fe050198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe0559af640 1 --2- 172.21.15.83:0/2439466573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0501053e0 0x7fe050198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe0559af640 1 -- 172.21.15.83:0/2439466573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe050107d10 msgr2=0x7fe05019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:28.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe0559af640 1 --2- 172.21.15.83:0/2439466573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe050107d10 0x7fe05019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe0559af640 1 -- 172.21.15.83:0/2439466573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05019fe30 con 0x7fe050102ab0 2024-04-25T11:43:28.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe0561b0640 1 --2- 172.21.15.83:0/2439466573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe050107d10 0x7fe05019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:28.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe0559af640 1 --2- 172.21.15.83:0/2439466573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe050102ab0 0x7fe050198210 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00c970 tx=0x7fe04c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:28.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe046ffd640 1 -- 172.21.15.83:0/2439466573 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c007bb0 con 0x7fe050102ab0 2024-04-25T11:43:28.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe046ffd640 1 -- 172.21.15.83:0/2439466573 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe04c007d50 con 0x7fe050102ab0 2024-04-25T11:43:28.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe057c13640 1 -- 172.21.15.83:0/2439466573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0501a0120 con 0x7fe050102ab0 2024-04-25T11:43:28.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe046ffd640 1 -- 172.21.15.83:0/2439466573 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c00f450 con 0x7fe050102ab0 2024-04-25T11:43:28.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.583+0000 7fe057c13640 1 -- 172.21.15.83:0/2439466573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0501a0640 con 0x7fe050102ab0 2024-04-25T11:43:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.584+0000 7fe057c13640 1 -- 172.21.15.83:0/2439466573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0500fdef0 con 0x7fe050102ab0 2024-04-25T11:43:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.585+0000 7fe046ffd640 1 -- 172.21.15.83:0/2439466573 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe04c004f20 con 0x7fe050102ab0 2024-04-25T11:43:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.586+0000 7fe046ffd640 1 --2- 172.21.15.83:0/2439466573 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe03c0a3c30 0x7fe03c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.586+0000 7fe0551ae640 1 --2- 172.21.15.83:0/2439466573 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe03c0a3c30 0x7fe03c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.586+0000 7fe046ffd640 1 -- 172.21.15.83:0/2439466573 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fe04c0d0180 con 0x7fe050102ab0 2024-04-25T11:43:28.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.586+0000 7fe0551ae640 1 --2- 172.21.15.83:0/2439466573 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe03c0a3c30 0x7fe03c0a60f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe040006010 tx=0x7fe0400314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:28.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.591+0000 7fe046ffd640 1 -- 172.21.15.83:0/2439466573 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe04c014030 con 0x7fe050102ab0 2024-04-25T11:43:28.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.769+0000 7fe057c13640 1 -- 172.21.15.83:0/2439466573 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe050062b50 con 0x7fe050102ab0 2024-04-25T11:43:28.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.770+0000 7fe046ffd640 1 -- 172.21.15.83:0/2439466573 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22581 (secure 0 0 0) 0x7fe04c09cb70 con 0x7fe050102ab0 2024-04-25T11:43:28.757 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:28.757 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:42:51.074495+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:41:42.546991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:41:43.764226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:43:28.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.772+0000 7fe057c13640 1 -- 172.21.15.83:0/2439466573 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe03c0a3c30 msgr2=0x7fe03c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:28.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.772+0000 7fe057c13640 1 --2- 172.21.15.83:0/2439466573 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe03c0a3c30 0x7fe03c0a60f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe040006010 tx=0x7fe0400314a0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.772+0000 7fe057c13640 1 -- 172.21.15.83:0/2439466573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe050102ab0 msgr2=0x7fe050198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:28.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.772+0000 7fe057c13640 1 --2- 172.21.15.83:0/2439466573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe050102ab0 0x7fe050198210 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00c970 tx=0x7fe04c00ce30 comp rx=0 tx=0).stop 2024-04-25T11:43:28.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.772+0000 7fe057c13640 1 -- 172.21.15.83:0/2439466573 shutdown_connections 2024-04-25T11:43:28.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.772+0000 7fe057c13640 1 --2- 172.21.15.83:0/2439466573 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fe03c0a3c30 0x7fe03c0a60f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.772+0000 7fe057c13640 1 --2- 172.21.15.83:0/2439466573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe050107d10 0x7fe05019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.772+0000 7fe057c13640 1 --2- 172.21.15.83:0/2439466573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0501053e0 0x7fe050198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.772+0000 7fe057c13640 1 --2- 172.21.15.83:0/2439466573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe050102ab0 0x7fe050198210 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.772+0000 7fe057c13640 1 -- 172.21.15.83:0/2439466573 >> 172.21.15.83:0/2439466573 conn(0x7fe0500fc400 msgr2=0x7fe050103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:28.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.772+0000 7fe057c13640 1 -- 172.21.15.83:0/2439466573 shutdown_connections 2024-04-25T11:43:28.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.772+0000 7fe057c13640 1 -- 172.21.15.83:0/2439466573 wait complete. 2024-04-25T11:43:28.770 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:37:15.914779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:41:42.546991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:41:43.764226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:43:28.770 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:43:28.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.884+0000 7f6a86ecb640 1 -- 172.21.15.83:0/168203211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a801029a0 msgr2=0x7f6a801092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:28.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.884+0000 7f6a86ecb640 1 --2- 172.21.15.83:0/168203211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a801029a0 0x7f6a801092c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009ca0 tx=0x7f6a7802a8c0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.884+0000 7f6a86ecb640 1 -- 172.21.15.83:0/168203211 shutdown_connections 2024-04-25T11:43:28.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.884+0000 7f6a86ecb640 1 --2- 172.21.15.83:0/168203211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a801029a0 0x7f6a801092c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.884+0000 7f6a86ecb640 1 --2- 172.21.15.83:0/168203211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a80101fe0 0x7f6a80102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.884+0000 7f6a86ecb640 1 --2- 172.21.15.83:0/168203211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a80100e90 0x7f6a801012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.884+0000 7f6a86ecb640 1 -- 172.21.15.83:0/168203211 >> 172.21.15.83:0/168203211 conn(0x7f6a800fc400 msgr2=0x7f6a800fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:28.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.884+0000 7f6a86ecb640 1 -- 172.21.15.83:0/168203211 shutdown_connections 2024-04-25T11:43:28.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.884+0000 7f6a86ecb640 1 -- 172.21.15.83:0/168203211 wait complete. 2024-04-25T11:43:28.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a86ecb640 1 Processor -- start 2024-04-25T11:43:28.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a86ecb640 1 -- start start 2024-04-25T11:43:28.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a86ecb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a80100e90 0x7f6a8019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:28.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a86ecb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a80101fe0 0x7f6a8019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:28.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a86ecb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a801029a0 0x7f6a801a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:28.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a86ecb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8010c9f0 con 0x7f6a80101fe0 2024-04-25T11:43:28.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a86ecb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8010c870 con 0x7f6a80100e90 2024-04-25T11:43:28.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a86ecb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8010cb70 con 0x7f6a801029a0 2024-04-25T11:43:28.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a7ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a80101fe0 0x7f6a8019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:28.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a7ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a80101fe0 0x7f6a8019a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48530/0 (socket says 172.21.15.83:48530) 2024-04-25T11:43:28.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a7ffff640 1 -- 172.21.15.83:0/1998627634 learned_addr learned my addr 172.21.15.83:0/1998627634 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:28.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a84c67640 1 --2- 172.21.15.83:0/1998627634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a80100e90 0x7f6a8019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:28.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a85468640 1 --2- 172.21.15.83:0/1998627634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a801029a0 0x7f6a801a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:28.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a7ffff640 1 -- 172.21.15.83:0/1998627634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a801029a0 msgr2=0x7f6a801a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:28.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a7ffff640 1 --2- 172.21.15.83:0/1998627634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a801029a0 0x7f6a801a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a7ffff640 1 -- 172.21.15.83:0/1998627634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a80100e90 msgr2=0x7f6a8019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:28.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a7ffff640 1 --2- 172.21.15.83:0/1998627634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a80100e90 0x7f6a8019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:28.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.885+0000 7f6a7ffff640 1 -- 172.21.15.83:0/1998627634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a801a2000 con 0x7f6a80101fe0 2024-04-25T11:43:28.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.886+0000 7f6a7ffff640 1 --2- 172.21.15.83:0/1998627634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a80101fe0 0x7f6a8019a920 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400d770 tx=0x7f6a7400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:28.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.886+0000 7f6a7dffb640 1 -- 172.21.15.83:0/1998627634 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74017070 con 0x7f6a80101fe0 2024-04-25T11:43:28.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.886+0000 7f6a7dffb640 1 -- 172.21.15.83:0/1998627634 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a74009c30 con 0x7f6a80101fe0 2024-04-25T11:43:28.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.886+0000 7f6a7dffb640 1 -- 172.21.15.83:0/1998627634 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74003cc0 con 0x7f6a80101fe0 2024-04-25T11:43:28.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.886+0000 7f6a86ecb640 1 -- 172.21.15.83:0/1998627634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a801a22f0 con 0x7f6a80101fe0 2024-04-25T11:43:28.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.886+0000 7f6a86ecb640 1 -- 172.21.15.83:0/1998627634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a801a2840 con 0x7f6a80101fe0 2024-04-25T11:43:28.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.887+0000 7f6a86ecb640 1 -- 172.21.15.83:0/1998627634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a801a2aa0 con 0x7f6a80101fe0 2024-04-25T11:43:28.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.888+0000 7f6a7dffb640 1 -- 172.21.15.83:0/1998627634 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6a74003e60 con 0x7f6a80101fe0 2024-04-25T11:43:28.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.888+0000 7f6a7dffb640 1 --2- 172.21.15.83:0/1998627634 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f6a680a3bb0 0x7f6a680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:28.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.889+0000 7f6a84c67640 1 --2- 172.21.15.83:0/1998627634 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f6a680a3bb0 0x7f6a680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:28.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.889+0000 7f6a7dffb640 1 -- 172.21.15.83:0/1998627634 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f6a740cfcd0 con 0x7f6a80101fe0 2024-04-25T11:43:28.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.889+0000 7f6a84c67640 1 --2- 172.21.15.83:0/1998627634 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f6a680a3bb0 0x7f6a680a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6a70005fd0 tx=0x7f6a7000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:28.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:28.893+0000 7f6a7dffb640 1 -- 172.21.15.83:0/1998627634 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a74009de0 con 0x7f6a80101fe0 2024-04-25T11:43:29.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.076+0000 7f6a86ecb640 1 -- 172.21.15.83:0/1998627634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6a801a2c70 con 0x7f6a80101fe0 2024-04-25T11:43:29.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.076+0000 7f6a7dffb640 1 -- 172.21.15.83:0/1998627634 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f6a7409c070 con 0x7f6a80101fe0 2024-04-25T11:43:29.063 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T11:43:29.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.078+0000 7f6a86ecb640 1 -- 172.21.15.83:0/1998627634 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f6a680a3bb0 msgr2=0x7f6a680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:29.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.078+0000 7f6a86ecb640 1 --2- 172.21.15.83:0/1998627634 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f6a680a3bb0 0x7f6a680a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6a70005fd0 tx=0x7f6a7000b390 comp rx=0 tx=0).stop 2024-04-25T11:43:29.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.078+0000 7f6a86ecb640 1 -- 172.21.15.83:0/1998627634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a80101fe0 msgr2=0x7f6a8019a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:29.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.078+0000 7f6a86ecb640 1 --2- 172.21.15.83:0/1998627634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a80101fe0 0x7f6a8019a920 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400d770 tx=0x7f6a7400dc30 comp rx=0 tx=0).stop 2024-04-25T11:43:29.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.078+0000 7f6a86ecb640 1 -- 172.21.15.83:0/1998627634 shutdown_connections 2024-04-25T11:43:29.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.078+0000 7f6a86ecb640 1 --2- 172.21.15.83:0/1998627634 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f6a680a3bb0 0x7f6a680a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.078+0000 7f6a86ecb640 1 --2- 172.21.15.83:0/1998627634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a80101fe0 0x7f6a8019a920 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.078+0000 7f6a86ecb640 1 --2- 172.21.15.83:0/1998627634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a801029a0 0x7f6a801a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.078+0000 7f6a86ecb640 1 --2- 172.21.15.83:0/1998627634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6a80100e90 0x7f6a8019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.078+0000 7f6a86ecb640 1 -- 172.21.15.83:0/1998627634 >> 172.21.15.83:0/1998627634 conn(0x7f6a800fc400 msgr2=0x7f6a800fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:29.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.079+0000 7f6a86ecb640 1 -- 172.21.15.83:0/1998627634 shutdown_connections 2024-04-25T11:43:29.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.079+0000 7f6a86ecb640 1 -- 172.21.15.83:0/1998627634 wait complete. 2024-04-25T11:43:29.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:43:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.190+0000 7f0ef827c640 1 -- 172.21.15.83:0/2452507752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef0102a90 msgr2=0x7f0ef0104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.190+0000 7f0ef827c640 1 --2- 172.21.15.83:0/2452507752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef0102a90 0x7f0ef0104e80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0009900 tx=0x7f0ee002a410 comp rx=0 tx=0).stop 2024-04-25T11:43:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.190+0000 7f0ef827c640 1 -- 172.21.15.83:0/2452507752 shutdown_connections 2024-04-25T11:43:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.190+0000 7f0ef827c640 1 --2- 172.21.15.83:0/2452507752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef0107cf0 0x7f0ef010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.190+0000 7f0ef827c640 1 --2- 172.21.15.83:0/2452507752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ef01053c0 0x7f0ef01077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.190+0000 7f0ef827c640 1 --2- 172.21.15.83:0/2452507752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef0102a90 0x7f0ef0104e80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.190+0000 7f0ef827c640 1 -- 172.21.15.83:0/2452507752 >> 172.21.15.83:0/2452507752 conn(0x7f0ef00fc400 msgr2=0x7f0ef00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:29.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.190+0000 7f0ef827c640 1 -- 172.21.15.83:0/2452507752 shutdown_connections 2024-04-25T11:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.190+0000 7f0ef827c640 1 -- 172.21.15.83:0/2452507752 wait complete. 2024-04-25T11:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.190+0000 7f0ef827c640 1 Processor -- start 2024-04-25T11:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.190+0000 7f0ef827c640 1 -- start start 2024-04-25T11:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.190+0000 7f0ef827c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ef0102a90 0x7f0ef0198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef827c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef01053c0 0x7f0ef0198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef827c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef0107cf0 0x7f0ef019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef827c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef010dad0 con 0x7f0ef0107cf0 2024-04-25T11:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef827c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef010d950 con 0x7f0ef01053c0 2024-04-25T11:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef827c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef010dc50 con 0x7f0ef0102a90 2024-04-25T11:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef6819640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef0107cf0 0x7f0ef019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef6819640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef0107cf0 0x7f0ef019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48538/0 (socket says 172.21.15.83:48538) 2024-04-25T11:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef6819640 1 -- 172.21.15.83:0/1293095938 learned_addr learned my addr 172.21.15.83:0/1293095938 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef6018640 1 --2- 172.21.15.83:0/1293095938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ef0102a90 0x7f0ef0198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:29.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef5817640 1 --2- 172.21.15.83:0/1293095938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef01053c0 0x7f0ef0198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef6819640 1 -- 172.21.15.83:0/1293095938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ef0102a90 msgr2=0x7f0ef0198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef6819640 1 --2- 172.21.15.83:0/1293095938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ef0102a90 0x7f0ef0198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef6819640 1 -- 172.21.15.83:0/1293095938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef01053c0 msgr2=0x7f0ef0198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef6819640 1 --2- 172.21.15.83:0/1293095938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef01053c0 0x7f0ef0198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef6819640 1 -- 172.21.15.83:0/1293095938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef019fe60 con 0x7f0ef0107cf0 2024-04-25T11:43:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef6018640 1 --2- 172.21.15.83:0/1293095938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ef0102a90 0x7f0ef0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef6819640 1 --2- 172.21.15.83:0/1293095938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef0107cf0 0x7f0ef019f7f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800eaa0 tx=0x7f0ee800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ee77fe640 1 -- 172.21.15.83:0/1293095938 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee800cdb0 con 0x7f0ef0107cf0 2024-04-25T11:43:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ee77fe640 1 -- 172.21.15.83:0/1293095938 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee8005020 con 0x7f0ef0107cf0 2024-04-25T11:43:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ee77fe640 1 -- 172.21.15.83:0/1293095938 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80116d0 con 0x7f0ef0107cf0 2024-04-25T11:43:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef827c640 1 -- 172.21.15.83:0/1293095938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef01a0150 con 0x7f0ef0107cf0 2024-04-25T11:43:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.191+0000 7f0ef827c640 1 -- 172.21.15.83:0/1293095938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef01a0650 con 0x7f0ef0107cf0 2024-04-25T11:43:29.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.192+0000 7f0ef827c640 1 -- 172.21.15.83:0/1293095938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef00fdef0 con 0x7f0ef0107cf0 2024-04-25T11:43:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.193+0000 7f0ee77fe640 1 -- 172.21.15.83:0/1293095938 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0ee80051d0 con 0x7f0ef0107cf0 2024-04-25T11:43:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.194+0000 7f0ee77fe640 1 --2- 172.21.15.83:0/1293095938 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0edc0a3bb0 0x7f0edc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.194+0000 7f0ef6018640 1 --2- 172.21.15.83:0/1293095938 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0edc0a3bb0 0x7f0edc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.194+0000 7f0ee77fe640 1 -- 172.21.15.83:0/1293095938 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f0ee80d05f0 con 0x7f0ef0107cf0 2024-04-25T11:43:29.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.194+0000 7f0ef6018640 1 --2- 172.21.15.83:0/1293095938 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0edc0a3bb0 0x7f0edc0a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0000c00 tx=0x7f0ee00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:29.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.198+0000 7f0ee77fe640 1 -- 172.21.15.83:0/1293095938 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ee8016030 con 0x7f0ef0107cf0 2024-04-25T11:43:29.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:29 smithi111 ceph-mon[32784]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-25T11:43:29.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:29 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2439466573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.377+0000 7f0ef827c640 1 -- 172.21.15.83:0/1293095938 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0ef0062b50 con 0x7f0ef0107cf0 2024-04-25T11:43:29.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.377+0000 7f0ee77fe640 1 -- 172.21.15.83:0/1293095938 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f0ee809cfe0 con 0x7f0ef0107cf0 2024-04-25T11:43:29.363 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T11:43:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.379+0000 7f0ef827c640 1 -- 172.21.15.83:0/1293095938 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0edc0a3bb0 msgr2=0x7f0edc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.379+0000 7f0ef827c640 1 --2- 172.21.15.83:0/1293095938 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0edc0a3bb0 0x7f0edc0a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0000c00 tx=0x7f0ee00314a0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.379+0000 7f0ef827c640 1 -- 172.21.15.83:0/1293095938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef0107cf0 msgr2=0x7f0ef019f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.379+0000 7f0ef827c640 1 --2- 172.21.15.83:0/1293095938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef0107cf0 0x7f0ef019f7f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800eaa0 tx=0x7f0ee800ef60 comp rx=0 tx=0).stop 2024-04-25T11:43:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.379+0000 7f0ef827c640 1 -- 172.21.15.83:0/1293095938 shutdown_connections 2024-04-25T11:43:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.379+0000 7f0ef827c640 1 --2- 172.21.15.83:0/1293095938 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0edc0a3bb0 0x7f0edc0a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.379+0000 7f0ef827c640 1 --2- 172.21.15.83:0/1293095938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ef0107cf0 0x7f0ef019f7f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.379+0000 7f0ef827c640 1 --2- 172.21.15.83:0/1293095938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ef01053c0 0x7f0ef0198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.379+0000 7f0ef827c640 1 --2- 172.21.15.83:0/1293095938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ef0102a90 0x7f0ef0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.379+0000 7f0ef827c640 1 -- 172.21.15.83:0/1293095938 >> 172.21.15.83:0/1293095938 conn(0x7f0ef00fc400 msgr2=0x7f0ef0104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.379+0000 7f0ef827c640 1 -- 172.21.15.83:0/1293095938 shutdown_connections 2024-04-25T11:43:29.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.379+0000 7f0ef827c640 1 -- 172.21.15.83:0/1293095938 wait complete. 2024-04-25T11:43:29.376 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:29 smithi083 ceph-mon[29330]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-25T11:43:29.376 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:29 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2439466573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:29.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:43:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.491+0000 7f9f3f944640 1 -- 172.21.15.83:0/1102774712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f38101fe0 msgr2=0x7f9f38102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.491+0000 7f9f3f944640 1 --2- 172.21.15.83:0/1102774712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f38101fe0 0x7f9f38102460 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9f28009900 tx=0x7f9f2802a410 comp rx=0 tx=0).stop 2024-04-25T11:43:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.491+0000 7f9f3f944640 1 -- 172.21.15.83:0/1102774712 shutdown_connections 2024-04-25T11:43:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.491+0000 7f9f3f944640 1 --2- 172.21.15.83:0/1102774712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f38101fe0 0x7f9f38102460 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.491+0000 7f9f3f944640 1 --2- 172.21.15.83:0/1102774712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f381029a0 0x7f9f381092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.491+0000 7f9f3f944640 1 --2- 172.21.15.83:0/1102774712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f38100e90 0x7f9f381012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.491+0000 7f9f3f944640 1 -- 172.21.15.83:0/1102774712 >> 172.21.15.83:0/1102774712 conn(0x7f9f380fc400 msgr2=0x7f9f380fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:29.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.491+0000 7f9f3f944640 1 -- 172.21.15.83:0/1102774712 shutdown_connections 2024-04-25T11:43:29.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.491+0000 7f9f3f944640 1 -- 172.21.15.83:0/1102774712 wait complete. 2024-04-25T11:43:29.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.491+0000 7f9f3f944640 1 Processor -- start 2024-04-25T11:43:29.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3f944640 1 -- start start 2024-04-25T11:43:29.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3f944640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f38100e90 0x7f9f3806b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:29.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3f944640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f38101fe0 0x7f9f3819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:29.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3f944640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f381029a0 0x7f9f3819f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:29.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3f944640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3810c9d0 con 0x7f9f381029a0 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3f944640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3810c850 con 0x7f9f38100e90 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3f944640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3810cb50 con 0x7f9f38101fe0 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3dee1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f381029a0 0x7f9f3819f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3dee1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f381029a0 0x7f9f3819f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48546/0 (socket says 172.21.15.83:48546) 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3dee1640 1 -- 172.21.15.83:0/3798570179 learned_addr learned my addr 172.21.15.83:0/3798570179 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3cedf640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f38101fe0 0x7f9f3819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3d6e0640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f38100e90 0x7f9f3806b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3dee1640 1 -- 172.21.15.83:0/3798570179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f38101fe0 msgr2=0x7f9f3819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3dee1640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f38101fe0 0x7f9f3819cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3dee1640 1 -- 172.21.15.83:0/3798570179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f38100e90 msgr2=0x7f9f3806b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3dee1640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f38100e90 0x7f9f3806b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3dee1640 1 -- 172.21.15.83:0/3798570179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3819fe60 con 0x7f9f381029a0 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3cedf640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f38101fe0 0x7f9f3819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3d6e0640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f38100e90 0x7f9f3806b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3dee1640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f381029a0 0x7f9f3819f550 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000eb10 tx=0x7f9f3000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:29.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f2e7fc640 1 -- 172.21.15.83:0/3798570179 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30004b20 con 0x7f9f381029a0 2024-04-25T11:43:29.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f2e7fc640 1 -- 172.21.15.83:0/3798570179 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f30004cc0 con 0x7f9f381029a0 2024-04-25T11:43:29.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3f944640 1 -- 172.21.15.83:0/3798570179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f381a00f0 con 0x7f9f381029a0 2024-04-25T11:43:29.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f2e7fc640 1 -- 172.21.15.83:0/3798570179 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30010670 con 0x7f9f381029a0 2024-04-25T11:43:29.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.492+0000 7f9f3f944640 1 -- 172.21.15.83:0/3798570179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f381a0610 con 0x7f9f381029a0 2024-04-25T11:43:29.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.493+0000 7f9f3f944640 1 -- 172.21.15.83:0/3798570179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f38108290 con 0x7f9f381029a0 2024-04-25T11:43:29.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.495+0000 7f9f2e7fc640 1 -- 172.21.15.83:0/3798570179 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9f30005000 con 0x7f9f381029a0 2024-04-25T11:43:29.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.495+0000 7f9f2e7fc640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9f240a3c30 0x7f9f240a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:29.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.495+0000 7f9f3d6e0640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9f240a3c30 0x7f9f240a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:29.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.495+0000 7f9f2e7fc640 1 -- 172.21.15.83:0/3798570179 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f9f300d0c50 con 0x7f9f381029a0 2024-04-25T11:43:29.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.496+0000 7f9f3d6e0640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9f240a3c30 0x7f9f240a60f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009b40 tx=0x7f9f34005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:29.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.499+0000 7f9f2e7fc640 1 -- 172.21.15.83:0/3798570179 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f30053ff0 con 0x7f9f381029a0 2024-04-25T11:43:29.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:29 smithi094 ceph-mon[33032]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-25T11:43:29.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:29 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2439466573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.679+0000 7f9f3f944640 1 -- 172.21.15.83:0/3798570179 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9f3802d830 con 0x7f9f381029a0 2024-04-25T11:43:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.679+0000 7f9f2e7fc640 1 -- 172.21.15.83:0/3798570179 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f9f3009d640 con 0x7f9f381029a0 2024-04-25T11:43:29.665 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T11:43:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.681+0000 7f9f3f944640 1 -- 172.21.15.83:0/3798570179 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9f240a3c30 msgr2=0x7f9f240a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.681+0000 7f9f3f944640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9f240a3c30 0x7f9f240a60f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009b40 tx=0x7f9f34005e50 comp rx=0 tx=0).stop 2024-04-25T11:43:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.681+0000 7f9f3f944640 1 -- 172.21.15.83:0/3798570179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f381029a0 msgr2=0x7f9f3819f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.681+0000 7f9f3f944640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f381029a0 0x7f9f3819f550 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000eb10 tx=0x7f9f3000c2d0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.681+0000 7f9f3f944640 1 -- 172.21.15.83:0/3798570179 shutdown_connections 2024-04-25T11:43:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.681+0000 7f9f3f944640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f9f240a3c30 0x7f9f240a60f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.681+0000 7f9f3f944640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f381029a0 0x7f9f3819f550 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.681+0000 7f9f3f944640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f38101fe0 0x7f9f3819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.681+0000 7f9f3f944640 1 --2- 172.21.15.83:0/3798570179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f38100e90 0x7f9f3806b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.681+0000 7f9f3f944640 1 -- 172.21.15.83:0/3798570179 >> 172.21.15.83:0/3798570179 conn(0x7f9f380fc400 msgr2=0x7f9f38105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.681+0000 7f9f3f944640 1 -- 172.21.15.83:0/3798570179 shutdown_connections 2024-04-25T11:43:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.681+0000 7f9f3f944640 1 -- 172.21.15.83:0/3798570179 wait complete. 2024-04-25T11:43:29.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:43:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.789+0000 7ff6afa7c640 1 -- 172.21.15.83:0/142228868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6a8101fe0 msgr2=0x7ff6a8102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.789+0000 7ff6afa7c640 1 --2- 172.21.15.83:0/142228868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6a8101fe0 0x7ff6a8102460 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009900 tx=0x7ff69c02a410 comp rx=0 tx=0).stop 2024-04-25T11:43:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.789+0000 7ff6afa7c640 1 -- 172.21.15.83:0/142228868 shutdown_connections 2024-04-25T11:43:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.789+0000 7ff6afa7c640 1 --2- 172.21.15.83:0/142228868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6a8101fe0 0x7ff6a8102460 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.789+0000 7ff6afa7c640 1 --2- 172.21.15.83:0/142228868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a81029a0 0x7ff6a81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.789+0000 7ff6afa7c640 1 --2- 172.21.15.83:0/142228868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a8100e90 0x7ff6a81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.789+0000 7ff6afa7c640 1 -- 172.21.15.83:0/142228868 >> 172.21.15.83:0/142228868 conn(0x7ff6a80fc400 msgr2=0x7ff6a80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.789+0000 7ff6afa7c640 1 -- 172.21.15.83:0/142228868 shutdown_connections 2024-04-25T11:43:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.789+0000 7ff6afa7c640 1 -- 172.21.15.83:0/142228868 wait complete. 2024-04-25T11:43:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6afa7c640 1 Processor -- start 2024-04-25T11:43:29.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6afa7c640 1 -- start start 2024-04-25T11:43:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6afa7c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a8100e90 0x7ff6a8197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6afa7c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6a8101fe0 0x7ff6a8197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6afa7c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a81029a0 0x7ff6a819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6afa7c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a810ca40 con 0x7ff6a8101fe0 2024-04-25T11:43:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6afa7c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a810c8c0 con 0x7ff6a8100e90 2024-04-25T11:43:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6afa7c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a810cbc0 con 0x7ff6a81029a0 2024-04-25T11:43:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6ad017640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6a8101fe0 0x7ff6a8197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6ad017640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6a8101fe0 0x7ff6a8197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48562/0 (socket says 172.21.15.83:48562) 2024-04-25T11:43:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6ad017640 1 -- 172.21.15.83:0/1734534708 learned_addr learned my addr 172.21.15.83:0/1734534708 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6ad818640 1 --2- 172.21.15.83:0/1734534708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a8100e90 0x7ff6a8197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6ad017640 1 -- 172.21.15.83:0/1734534708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a81029a0 msgr2=0x7ff6a819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6ad017640 1 --2- 172.21.15.83:0/1734534708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a81029a0 0x7ff6a819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6ad017640 1 -- 172.21.15.83:0/1734534708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a8100e90 msgr2=0x7ff6a8197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6ad017640 1 --2- 172.21.15.83:0/1734534708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a8100e90 0x7ff6a8197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:29.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6ad017640 1 -- 172.21.15.83:0/1734534708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a819ed60 con 0x7ff6a8101fe0 2024-04-25T11:43:29.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.790+0000 7ff6ad818640 1 --2- 172.21.15.83:0/1734534708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a8100e90 0x7ff6a8197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:29.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.791+0000 7ff6ad017640 1 --2- 172.21.15.83:0/1734534708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6a8101fe0 0x7ff6a8197680 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff69c006010 tx=0x7ff69c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:29.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.791+0000 7ff6a6ffd640 1 -- 172.21.15.83:0/1734534708 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c046070 con 0x7ff6a8101fe0 2024-04-25T11:43:29.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.791+0000 7ff6a6ffd640 1 -- 172.21.15.83:0/1734534708 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff69c004490 con 0x7ff6a8101fe0 2024-04-25T11:43:29.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.791+0000 7ff6afa7c640 1 -- 172.21.15.83:0/1734534708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a80690e0 con 0x7ff6a8101fe0 2024-04-25T11:43:29.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.791+0000 7ff6a6ffd640 1 -- 172.21.15.83:0/1734534708 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c035050 con 0x7ff6a8101fe0 2024-04-25T11:43:29.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.791+0000 7ff6afa7c640 1 -- 172.21.15.83:0/1734534708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a8069650 con 0x7ff6a8101fe0 2024-04-25T11:43:29.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.792+0000 7ff6afa7c640 1 -- 172.21.15.83:0/1734534708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a8068d70 con 0x7ff6a8101fe0 2024-04-25T11:43:29.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.793+0000 7ff6a6ffd640 1 -- 172.21.15.83:0/1734534708 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff69c049070 con 0x7ff6a8101fe0 2024-04-25T11:43:29.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.794+0000 7ff6a6ffd640 1 --2- 172.21.15.83:0/1734534708 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff6900a3bb0 0x7ff6900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:29.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.794+0000 7ff6ad818640 1 --2- 172.21.15.83:0/1734534708 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff6900a3bb0 0x7ff6900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:29.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.794+0000 7ff6a6ffd640 1 -- 172.21.15.83:0/1734534708 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7ff69c0f24e0 con 0x7ff6a8101fe0 2024-04-25T11:43:29.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.794+0000 7ff6ad818640 1 --2- 172.21.15.83:0/1734534708 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff6900a3bb0 0x7ff6900a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0006fd0 tx=0x7ff6a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:29.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:29.797+0000 7ff6a6ffd640 1 -- 172.21.15.83:0/1734534708 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff69c050020 con 0x7ff6a8101fe0 2024-04-25T11:43:29.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.012+0000 7ff6afa7c640 1 -- 172.21.15.83:0/1734534708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6a8107e90 con 0x7ff6a8101fe0 2024-04-25T11:43:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.012+0000 7ff6a6ffd640 1 -- 172.21.15.83:0/1734534708 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7ff69c0beed0 con 0x7ff6a8101fe0 2024-04-25T11:43:29.999 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:29.999 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:41:44.787926+0000","modified":"2024-04-25T11:41:51.578220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24469,"mds_2":24442,"mds_3":24440,"mds_4":24430},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/1568064835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1568064835},{"type":"v1","addr":"172.21.15.83:6839","nonce":1568064835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/1576544313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1576544313},{"type":"v1","addr":"172.21.15.83:6841","nonce":1576544313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":4,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.94:6837/2475976592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2475976592},{"type":"v1","addr":"172.21.15.94:6837","nonce":2475976592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.111:6837/2927430878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2927430878},{"type":"v1","addr":"172.21.15.111:6837","nonce":2927430878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/2233531151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2233531151},{"type":"v1","addr":"172.21.15.111:6839","nonce":2233531151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/1461078471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1461078471},{"type":"v1","addr":"172.21.15.94:6839","nonce":1461078471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24440,24430,24479,24469,24442]},"id":1}]} 2024-04-25T11:43:29.999 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-04-25T11:43:30.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.015+0000 7ff6afa7c640 1 -- 172.21.15.83:0/1734534708 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff6900a3bb0 msgr2=0x7ff6900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.015+0000 7ff6afa7c640 1 --2- 172.21.15.83:0/1734534708 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff6900a3bb0 0x7ff6900a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0006fd0 tx=0x7ff6a0008040 comp rx=0 tx=0).stop 2024-04-25T11:43:30.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.015+0000 7ff6afa7c640 1 -- 172.21.15.83:0/1734534708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6a8101fe0 msgr2=0x7ff6a8197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.015+0000 7ff6afa7c640 1 --2- 172.21.15.83:0/1734534708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6a8101fe0 0x7ff6a8197680 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff69c006010 tx=0x7ff69c004270 comp rx=0 tx=0).stop 2024-04-25T11:43:30.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.015+0000 7ff6afa7c640 1 -- 172.21.15.83:0/1734534708 shutdown_connections 2024-04-25T11:43:30.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.015+0000 7ff6afa7c640 1 --2- 172.21.15.83:0/1734534708 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7ff6900a3bb0 0x7ff6900a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.015+0000 7ff6afa7c640 1 --2- 172.21.15.83:0/1734534708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6a8101fe0 0x7ff6a8197680 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.015+0000 7ff6afa7c640 1 --2- 172.21.15.83:0/1734534708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a81029a0 0x7ff6a819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.015+0000 7ff6afa7c640 1 --2- 172.21.15.83:0/1734534708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6a8100e90 0x7ff6a8197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.015+0000 7ff6afa7c640 1 -- 172.21.15.83:0/1734534708 >> 172.21.15.83:0/1734534708 conn(0x7ff6a80fc400 msgr2=0x7ff6a8105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:30.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.015+0000 7ff6afa7c640 1 -- 172.21.15.83:0/1734534708 shutdown_connections 2024-04-25T11:43:30.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.015+0000 7ff6afa7c640 1 -- 172.21.15.83:0/1734534708 wait complete. 2024-04-25T11:43:30.013 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:43:30.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.125+0000 7f00be9ad640 1 -- 172.21.15.83:0/2002963074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b8101fc0 msgr2=0x7f00b81023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.125+0000 7f00be9ad640 1 --2- 172.21.15.83:0/2002963074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b8101fc0 0x7f00b81023e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009900 tx=0x7f00a802a410 comp rx=0 tx=0).stop 2024-04-25T11:43:30.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.125+0000 7f00be9ad640 1 -- 172.21.15.83:0/2002963074 shutdown_connections 2024-04-25T11:43:30.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.125+0000 7f00be9ad640 1 --2- 172.21.15.83:0/2002963074 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b8103b70 0x7f00b810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.125+0000 7f00be9ad640 1 --2- 172.21.15.83:0/2002963074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00b81031b0 0x7f00b8103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.125+0000 7f00be9ad640 1 --2- 172.21.15.83:0/2002963074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b8101fc0 0x7f00b81023e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.125+0000 7f00be9ad640 1 -- 172.21.15.83:0/2002963074 >> 172.21.15.83:0/2002963074 conn(0x7f00b80fd550 msgr2=0x7f00b80ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:30.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.125+0000 7f00be9ad640 1 -- 172.21.15.83:0/2002963074 shutdown_connections 2024-04-25T11:43:30.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.125+0000 7f00be9ad640 1 -- 172.21.15.83:0/2002963074 wait complete. 2024-04-25T11:43:30.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00be9ad640 1 Processor -- start 2024-04-25T11:43:30.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00be9ad640 1 -- start start 2024-04-25T11:43:30.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00be9ad640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00b8101fc0 0x7f00b8075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:30.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00be9ad640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b81031b0 0x7f00b8074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:30.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00be9ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b8103b70 0x7f00b8074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:30.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00be9ad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f00b810db00 con 0x7f00b8103b70 2024-04-25T11:43:30.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00be9ad640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00b810d980 con 0x7f00b81031b0 2024-04-25T11:43:30.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00be9ad640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f00b810dc80 con 0x7f00b8101fc0 2024-04-25T11:43:30.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00bcf4a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b8103b70 0x7f00b8074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:30.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00bcf4a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b8103b70 0x7f00b8074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48588/0 (socket says 172.21.15.83:48588) 2024-04-25T11:43:30.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00bcf4a640 1 -- 172.21.15.83:0/780209879 learned_addr learned my addr 172.21.15.83:0/780209879 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:30.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00b77fe640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b81031b0 0x7f00b8074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:30.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00b7fff640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00b8101fc0 0x7f00b8075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:30.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00bcf4a640 1 -- 172.21.15.83:0/780209879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00b8101fc0 msgr2=0x7f00b8075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00bcf4a640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00b8101fc0 0x7f00b8075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00bcf4a640 1 -- 172.21.15.83:0/780209879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b81031b0 msgr2=0x7f00b8074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00bcf4a640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b81031b0 0x7f00b8074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00bcf4a640 1 -- 172.21.15.83:0/780209879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b8075110 con 0x7f00b8103b70 2024-04-25T11:43:30.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00b7fff640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00b8101fc0 0x7f00b8075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:30.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.126+0000 7f00b77fe640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b81031b0 0x7f00b8074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:30.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.127+0000 7f00bcf4a640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b8103b70 0x7f00b8074880 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f00b000cad0 tx=0x7f00b000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:30.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.127+0000 7f00b57fa640 1 -- 172.21.15.83:0/780209879 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b000ae20 con 0x7f00b8103b70 2024-04-25T11:43:30.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.127+0000 7f00be9ad640 1 -- 172.21.15.83:0/780209879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b81a22b0 con 0x7f00b8103b70 2024-04-25T11:43:30.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.127+0000 7f00b57fa640 1 -- 172.21.15.83:0/780209879 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00b0005020 con 0x7f00b8103b70 2024-04-25T11:43:30.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.127+0000 7f00b57fa640 1 -- 172.21.15.83:0/780209879 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b000f6b0 con 0x7f00b8103b70 2024-04-25T11:43:30.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.127+0000 7f00be9ad640 1 -- 172.21.15.83:0/780209879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b81a2800 con 0x7f00b8103b70 2024-04-25T11:43:30.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.128+0000 7f00be9ad640 1 -- 172.21.15.83:0/780209879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b8109380 con 0x7f00b8103b70 2024-04-25T11:43:30.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.129+0000 7f00b57fa640 1 -- 172.21.15.83:0/780209879 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f00b00051d0 con 0x7f00b8103b70 2024-04-25T11:43:30.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.129+0000 7f00b57fa640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f00a00a3bb0 0x7f00a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:30.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.129+0000 7f00b7fff640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f00a00a3bb0 0x7f00a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:30.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.129+0000 7f00b57fa640 1 -- 172.21.15.83:0/780209879 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f00b00d0560 con 0x7f00b8103b70 2024-04-25T11:43:30.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.130+0000 7f00b7fff640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f00a00a3bb0 0x7f00a00a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f00a8006010 tx=0x7f00a8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:30.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.134+0000 7f00b57fa640 1 -- 172.21.15.83:0/780209879 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f00b0014030 con 0x7f00b8103b70 2024-04-25T11:43:30.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.312+0000 7f00be9ad640 1 -- 172.21.15.83:0/780209879 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00b8062b50 con 0x7f00b8103b70 2024-04-25T11:43:30.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.314+0000 7f00b57fa640 1 -- 172.21.15.83:0/780209879 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22581 (secure 0 0 0) 0x7f00b009cf50 con 0x7f00b8103b70 2024-04-25T11:43:30.300 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:30.300 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:42:51.074495+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:41:42.546991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:41:43.764226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:43:30.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.316+0000 7f00be9ad640 1 -- 172.21.15.83:0/780209879 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f00a00a3bb0 msgr2=0x7f00a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.316+0000 7f00be9ad640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f00a00a3bb0 0x7f00a00a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f00a8006010 tx=0x7f00a8000bc0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.316+0000 7f00be9ad640 1 -- 172.21.15.83:0/780209879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b8103b70 msgr2=0x7f00b8074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.316+0000 7f00be9ad640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b8103b70 0x7f00b8074880 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f00b000cad0 tx=0x7f00b000cf90 comp rx=0 tx=0).stop 2024-04-25T11:43:30.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.316+0000 7f00be9ad640 1 -- 172.21.15.83:0/780209879 shutdown_connections 2024-04-25T11:43:30.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.316+0000 7f00be9ad640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f00a00a3bb0 0x7f00a00a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.316+0000 7f00be9ad640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00b8103b70 0x7f00b8074880 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.316+0000 7f00be9ad640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b81031b0 0x7f00b8074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.316+0000 7f00be9ad640 1 --2- 172.21.15.83:0/780209879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00b8101fc0 0x7f00b8075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.316+0000 7f00be9ad640 1 -- 172.21.15.83:0/780209879 >> 172.21.15.83:0/780209879 conn(0x7f00b80fd550 msgr2=0x7f00b8106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:30.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.316+0000 7f00be9ad640 1 -- 172.21.15.83:0/780209879 shutdown_connections 2024-04-25T11:43:30.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.316+0000 7f00be9ad640 1 -- 172.21.15.83:0/780209879 wait complete. 2024-04-25T11:43:30.312 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:30 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1998627634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:43:30.312 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:30 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1293095938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:43:30.312 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:30 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3798570179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:43:30.312 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:30 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1734534708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:43:30.314 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:37:15.914779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:41:42.546991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:41:43.764226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:43:30.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:43:30.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:30 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1998627634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:43:30.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:30 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1293095938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:43:30.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:30 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3798570179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:43:30.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:30 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1734534708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:43:30.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.429+0000 7fd853d20640 1 -- 172.21.15.83:0/112386931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd84c102ad0 msgr2=0x7fd84c104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.429+0000 7fd853d20640 1 --2- 172.21.15.83:0/112386931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd84c102ad0 0x7fd84c104ec0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd848009900 tx=0x7fd84802a410 comp rx=0 tx=0).stop 2024-04-25T11:43:30.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.429+0000 7fd853d20640 1 -- 172.21.15.83:0/112386931 shutdown_connections 2024-04-25T11:43:30.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.429+0000 7fd853d20640 1 --2- 172.21.15.83:0/112386931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd84c107d30 0x7fd84c10a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.429+0000 7fd853d20640 1 --2- 172.21.15.83:0/112386931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd84c105400 0x7fd84c1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.429+0000 7fd853d20640 1 --2- 172.21.15.83:0/112386931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd84c102ad0 0x7fd84c104ec0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.429+0000 7fd853d20640 1 -- 172.21.15.83:0/112386931 >> 172.21.15.83:0/112386931 conn(0x7fd84c0fc440 msgr2=0x7fd84c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:30.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.429+0000 7fd853d20640 1 -- 172.21.15.83:0/112386931 shutdown_connections 2024-04-25T11:43:30.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.429+0000 7fd853d20640 1 -- 172.21.15.83:0/112386931 wait complete. 2024-04-25T11:43:30.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd853d20640 1 Processor -- start 2024-04-25T11:43:30.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd853d20640 1 -- start start 2024-04-25T11:43:30.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd853d20640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd84c102ad0 0x7fd84c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:30.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd853d20640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd84c105400 0x7fd84c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:30.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd853d20640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd84c107d30 0x7fd84c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:30.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd853d20640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c10db10 con 0x7fd84c102ad0 2024-04-25T11:43:30.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd853d20640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c10d990 con 0x7fd84c105400 2024-04-25T11:43:30.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd853d20640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c10dc90 con 0x7fd84c107d30 2024-04-25T11:43:30.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd851abc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd84c102ad0 0x7fd84c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:30.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd851abc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd84c102ad0 0x7fd84c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48598/0 (socket says 172.21.15.83:48598) 2024-04-25T11:43:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd851abc640 1 -- 172.21.15.83:0/1401689950 learned_addr learned my addr 172.21.15.83:0/1401689950 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd8522bd640 1 --2- 172.21.15.83:0/1401689950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd84c107d30 0x7fd84c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd851abc640 1 -- 172.21.15.83:0/1401689950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd84c107d30 msgr2=0x7fd84c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd851abc640 1 --2- 172.21.15.83:0/1401689950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd84c107d30 0x7fd84c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd851abc640 1 -- 172.21.15.83:0/1401689950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd84c105400 msgr2=0x7fd84c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd851abc640 1 --2- 172.21.15.83:0/1401689950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd84c105400 0x7fd84c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd851abc640 1 -- 172.21.15.83:0/1401689950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84c19fe80 con 0x7fd84c102ad0 2024-04-25T11:43:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.430+0000 7fd851abc640 1 --2- 172.21.15.83:0/1401689950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd84c102ad0 0x7fd84c198260 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd848005940 tx=0x7fd848004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.431+0000 7fd842ffd640 1 -- 172.21.15.83:0/1401689950 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84803f030 con 0x7fd84c102ad0 2024-04-25T11:43:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.431+0000 7fd853d20640 1 -- 172.21.15.83:0/1401689950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84c1a0110 con 0x7fd84c102ad0 2024-04-25T11:43:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.431+0000 7fd842ffd640 1 -- 172.21.15.83:0/1401689950 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd848003cb0 con 0x7fd84c102ad0 2024-04-25T11:43:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.431+0000 7fd842ffd640 1 -- 172.21.15.83:0/1401689950 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd848035070 con 0x7fd84c102ad0 2024-04-25T11:43:30.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.431+0000 7fd853d20640 1 -- 172.21.15.83:0/1401689950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84c1a0600 con 0x7fd84c102ad0 2024-04-25T11:43:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.432+0000 7fd853d20640 1 -- 172.21.15.83:0/1401689950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd84c0fdf30 con 0x7fd84c102ad0 2024-04-25T11:43:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.433+0000 7fd842ffd640 1 -- 172.21.15.83:0/1401689950 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd848042030 con 0x7fd84c102ad0 2024-04-25T11:43:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.433+0000 7fd842ffd640 1 --2- 172.21.15.83:0/1401689950 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd8380a3bb0 0x7fd8380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.433+0000 7fd8512bb640 1 --2- 172.21.15.83:0/1401689950 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd8380a3bb0 0x7fd8380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.433+0000 7fd842ffd640 1 -- 172.21.15.83:0/1401689950 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd8480f0df0 con 0x7fd84c102ad0 2024-04-25T11:43:30.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.433+0000 7fd8512bb640 1 --2- 172.21.15.83:0/1401689950 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd8380a3bb0 0x7fd8380a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd83c005fd0 tx=0x7fd83c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:30.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.438+0000 7fd842ffd640 1 -- 172.21.15.83:0/1401689950 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd848003e60 con 0x7fd84c102ad0 2024-04-25T11:43:30.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:30 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1998627634' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:43:30.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:30 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1293095938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:43:30.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:30 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3798570179' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:43:30.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:30 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1734534708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:43:30.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.614+0000 7fd853d20640 1 -- 172.21.15.83:0/1401689950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd84c062b80 con 0x7fd84c102ad0 2024-04-25T11:43:30.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.615+0000 7fd842ffd640 1 -- 172.21.15.83:0/1401689950 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fd8480bd7e0 con 0x7fd84c102ad0 2024-04-25T11:43:30.601 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T11:43:30.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.616+0000 7fd853d20640 1 -- 172.21.15.83:0/1401689950 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd8380a3bb0 msgr2=0x7fd8380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.617+0000 7fd853d20640 1 --2- 172.21.15.83:0/1401689950 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd8380a3bb0 0x7fd8380a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd83c005fd0 tx=0x7fd83c005850 comp rx=0 tx=0).stop 2024-04-25T11:43:30.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.617+0000 7fd853d20640 1 -- 172.21.15.83:0/1401689950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd84c102ad0 msgr2=0x7fd84c198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.617+0000 7fd853d20640 1 --2- 172.21.15.83:0/1401689950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd84c102ad0 0x7fd84c198260 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd848005940 tx=0x7fd848004430 comp rx=0 tx=0).stop 2024-04-25T11:43:30.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.617+0000 7fd853d20640 1 -- 172.21.15.83:0/1401689950 shutdown_connections 2024-04-25T11:43:30.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.617+0000 7fd853d20640 1 --2- 172.21.15.83:0/1401689950 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd8380a3bb0 0x7fd8380a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.617+0000 7fd853d20640 1 --2- 172.21.15.83:0/1401689950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd84c107d30 0x7fd84c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.617+0000 7fd853d20640 1 --2- 172.21.15.83:0/1401689950 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd84c105400 0x7fd84c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.617+0000 7fd853d20640 1 --2- 172.21.15.83:0/1401689950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd84c102ad0 0x7fd84c198260 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.617+0000 7fd853d20640 1 -- 172.21.15.83:0/1401689950 >> 172.21.15.83:0/1401689950 conn(0x7fd84c0fc440 msgr2=0x7fd84c1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:30.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.617+0000 7fd853d20640 1 -- 172.21.15.83:0/1401689950 shutdown_connections 2024-04-25T11:43:30.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.617+0000 7fd853d20640 1 -- 172.21.15.83:0/1401689950 wait complete. 2024-04-25T11:43:30.614 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:43:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.728+0000 7f52055e5640 1 -- 172.21.15.83:0/608871145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5200103170 msgr2=0x7f52001035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.728+0000 7f52055e5640 1 --2- 172.21.15.83:0/608871145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5200103170 0x7f52001035f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009900 tx=0x7f51e802a410 comp rx=0 tx=0).stop 2024-04-25T11:43:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.728+0000 7f52055e5640 1 -- 172.21.15.83:0/608871145 shutdown_connections 2024-04-25T11:43:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.728+0000 7f52055e5640 1 --2- 172.21.15.83:0/608871145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5200103170 0x7f52001035f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.728+0000 7f52055e5640 1 --2- 172.21.15.83:0/608871145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5200103b30 0x7f520010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.728+0000 7f52055e5640 1 --2- 172.21.15.83:0/608871145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5200101f80 0x7f52001023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.728+0000 7f52055e5640 1 -- 172.21.15.83:0/608871145 >> 172.21.15.83:0/608871145 conn(0x7f52000fd510 msgr2=0x7f52000ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.728+0000 7f52055e5640 1 -- 172.21.15.83:0/608871145 shutdown_connections 2024-04-25T11:43:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.728+0000 7f52055e5640 1 -- 172.21.15.83:0/608871145 wait complete. 2024-04-25T11:43:30.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f52055e5640 1 Processor -- start 2024-04-25T11:43:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f52055e5640 1 -- start start 2024-04-25T11:43:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f52055e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5200101f80 0x7f5200198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f52055e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5200103170 0x7f52001986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f52055e5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5200103b30 0x7f520019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f52055e5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f520010dac0 con 0x7f5200101f80 2024-04-25T11:43:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f52055e5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f520010d940 con 0x7f5200103170 2024-04-25T11:43:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f52055e5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f520010dc40 con 0x7f5200103b30 2024-04-25T11:43:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f51feffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5200101f80 0x7f5200198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f51feffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5200101f80 0x7f5200198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48616/0 (socket says 172.21.15.83:48616) 2024-04-25T11:43:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f51feffd640 1 -- 172.21.15.83:0/2393390472 learned_addr learned my addr 172.21.15.83:0/2393390472 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f51ff7fe640 1 --2- 172.21.15.83:0/2393390472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5200103b30 0x7f520019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f51feffd640 1 -- 172.21.15.83:0/2393390472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5200103b30 msgr2=0x7f520019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f51feffd640 1 --2- 172.21.15.83:0/2393390472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5200103b30 0x7f520019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f51feffd640 1 -- 172.21.15.83:0/2393390472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5200103170 msgr2=0x7f52001986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f51f7fff640 1 --2- 172.21.15.83:0/2393390472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5200103170 0x7f52001986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:30.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f51feffd640 1 --2- 172.21.15.83:0/2393390472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5200103170 0x7f52001986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f51feffd640 1 -- 172.21.15.83:0/2393390472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520019fd90 con 0x7f5200101f80 2024-04-25T11:43:30.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f51ff7fe640 1 --2- 172.21.15.83:0/2393390472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5200103b30 0x7f520019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:30.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f51feffd640 1 --2- 172.21.15.83:0/2393390472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5200101f80 0x7f5200198170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f51f000e9c0 tx=0x7f51f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:30.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f51fcff9640 1 -- 172.21.15.83:0/2393390472 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f000cd80 con 0x7f5200101f80 2024-04-25T11:43:30.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.729+0000 7f51fcff9640 1 -- 172.21.15.83:0/2393390472 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f00035f0 con 0x7f5200101f80 2024-04-25T11:43:30.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.730+0000 7f52055e5640 1 -- 172.21.15.83:0/2393390472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52001a0080 con 0x7f5200101f80 2024-04-25T11:43:30.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.730+0000 7f51fcff9640 1 -- 172.21.15.83:0/2393390472 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0010690 con 0x7f5200101f80 2024-04-25T11:43:30.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.730+0000 7f52055e5640 1 -- 172.21.15.83:0/2393390472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52001a05d0 con 0x7f5200101f80 2024-04-25T11:43:30.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.730+0000 7f52055e5640 1 -- 172.21.15.83:0/2393390472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52001092f0 con 0x7f5200101f80 2024-04-25T11:43:30.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.731+0000 7f51fcff9640 1 -- 172.21.15.83:0/2393390472 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f51f00037a0 con 0x7f5200101f80 2024-04-25T11:43:30.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.732+0000 7f51fcff9640 1 --2- 172.21.15.83:0/2393390472 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f51ec0a3bb0 0x7f51ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:30.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.732+0000 7f51f7fff640 1 --2- 172.21.15.83:0/2393390472 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f51ec0a3bb0 0x7f51ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:30.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.732+0000 7f51fcff9640 1 -- 172.21.15.83:0/2393390472 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f51f00d08c0 con 0x7f5200101f80 2024-04-25T11:43:30.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.733+0000 7f51f7fff640 1 --2- 172.21.15.83:0/2393390472 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f51ec0a3bb0 0x7f51ec0a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f51e802a930 tx=0x7f51e8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:30.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.737+0000 7f51fcff9640 1 -- 172.21.15.83:0/2393390472 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51f009d2b0 con 0x7f5200101f80 2024-04-25T11:43:30.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.914+0000 7f52055e5640 1 -- 172.21.15.83:0/2393390472 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5200062b80 con 0x7f5200101f80 2024-04-25T11:43:30.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.915+0000 7f51fcff9640 1 -- 172.21.15.83:0/2393390472 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f51f009cbb0 con 0x7f5200101f80 2024-04-25T11:43:30.901 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T11:43:30.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.916+0000 7f52055e5640 1 -- 172.21.15.83:0/2393390472 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f51ec0a3bb0 msgr2=0x7f51ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.916+0000 7f52055e5640 1 --2- 172.21.15.83:0/2393390472 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f51ec0a3bb0 0x7f51ec0a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f51e802a930 tx=0x7f51e8000bc0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.916+0000 7f52055e5640 1 -- 172.21.15.83:0/2393390472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5200101f80 msgr2=0x7f5200198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:30.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.916+0000 7f52055e5640 1 --2- 172.21.15.83:0/2393390472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5200101f80 0x7f5200198170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f51f000e9c0 tx=0x7f51f000ee80 comp rx=0 tx=0).stop 2024-04-25T11:43:30.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.916+0000 7f52055e5640 1 -- 172.21.15.83:0/2393390472 shutdown_connections 2024-04-25T11:43:30.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.916+0000 7f52055e5640 1 --2- 172.21.15.83:0/2393390472 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f51ec0a3bb0 0x7f51ec0a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.916+0000 7f52055e5640 1 --2- 172.21.15.83:0/2393390472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5200103b30 0x7f520019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.916+0000 7f52055e5640 1 --2- 172.21.15.83:0/2393390472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5200103170 0x7f52001986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.916+0000 7f52055e5640 1 --2- 172.21.15.83:0/2393390472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5200101f80 0x7f5200198170 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:30.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.917+0000 7f52055e5640 1 -- 172.21.15.83:0/2393390472 >> 172.21.15.83:0/2393390472 conn(0x7f52000fd510 msgr2=0x7f52001064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:30.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.917+0000 7f52055e5640 1 -- 172.21.15.83:0/2393390472 shutdown_connections 2024-04-25T11:43:30.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:30.917+0000 7f52055e5640 1 -- 172.21.15.83:0/2393390472 wait complete. 2024-04-25T11:43:30.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:43:31.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.027+0000 7fdd28abb640 1 -- 172.21.15.83:0/3774969032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd241029a0 msgr2=0x7fdd241092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.027+0000 7fdd28abb640 1 --2- 172.21.15.83:0/3774969032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd241029a0 0x7fdd241092c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009ca0 tx=0x7fdd1c02a8c0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.027+0000 7fdd28abb640 1 -- 172.21.15.83:0/3774969032 shutdown_connections 2024-04-25T11:43:31.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.027+0000 7fdd28abb640 1 --2- 172.21.15.83:0/3774969032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd241029a0 0x7fdd241092c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.027+0000 7fdd28abb640 1 --2- 172.21.15.83:0/3774969032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd24101fe0 0x7fdd24102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.027+0000 7fdd28abb640 1 --2- 172.21.15.83:0/3774969032 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd24100e90 0x7fdd241012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.027+0000 7fdd28abb640 1 -- 172.21.15.83:0/3774969032 >> 172.21.15.83:0/3774969032 conn(0x7fdd240fc440 msgr2=0x7fdd240fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:31.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.027+0000 7fdd28abb640 1 -- 172.21.15.83:0/3774969032 shutdown_connections 2024-04-25T11:43:31.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.027+0000 7fdd28abb640 1 -- 172.21.15.83:0/3774969032 wait complete. 2024-04-25T11:43:31.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd28abb640 1 Processor -- start 2024-04-25T11:43:31.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd28abb640 1 -- start start 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd28abb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd24100e90 0x7fdd2406b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd28abb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd24101fe0 0x7fdd2419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd28abb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd241029a0 0x7fdd2419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd28abb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2410c9e0 con 0x7fdd241029a0 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd28abb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2410c860 con 0x7fdd24101fe0 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd28abb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2410cb60 con 0x7fdd24100e90 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd22d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd241029a0 0x7fdd2419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd22d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd241029a0 0x7fdd2419f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48642/0 (socket says 172.21.15.83:48642) 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd22d9d640 1 -- 172.21.15.83:0/835315641 learned_addr learned my addr 172.21.15.83:0/835315641 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd21d9b640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd24101fe0 0x7fdd2419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd2259c640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd24100e90 0x7fdd2406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd22d9d640 1 -- 172.21.15.83:0/835315641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd24100e90 msgr2=0x7fdd2406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd22d9d640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd24100e90 0x7fdd2406b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd22d9d640 1 -- 172.21.15.83:0/835315641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd24101fe0 msgr2=0x7fdd2419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd22d9d640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd24101fe0 0x7fdd2419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd22d9d640 1 -- 172.21.15.83:0/835315641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2419fdb0 con 0x7fdd241029a0 2024-04-25T11:43:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd2259c640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd24100e90 0x7fdd2406b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.028+0000 7fdd21d9b640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd24101fe0 0x7fdd2419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.029+0000 7fdd22d9d640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd241029a0 0x7fdd2419f4f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0097b0 tx=0x7fdd1c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.029+0000 7fdd137fe640 1 -- 172.21.15.83:0/835315641 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c004240 con 0x7fdd241029a0 2024-04-25T11:43:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.029+0000 7fdd137fe640 1 -- 172.21.15.83:0/835315641 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd1c0317c0 con 0x7fdd241029a0 2024-04-25T11:43:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.029+0000 7fdd28abb640 1 -- 172.21.15.83:0/835315641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd241a0040 con 0x7fdd241029a0 2024-04-25T11:43:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.029+0000 7fdd137fe640 1 -- 172.21.15.83:0/835315641 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c02fd80 con 0x7fdd241029a0 2024-04-25T11:43:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.029+0000 7fdd28abb640 1 -- 172.21.15.83:0/835315641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd241a0590 con 0x7fdd241029a0 2024-04-25T11:43:31.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.030+0000 7fdd28abb640 1 -- 172.21.15.83:0/835315641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd241081a0 con 0x7fdd241029a0 2024-04-25T11:43:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.031+0000 7fdd137fe640 1 -- 172.21.15.83:0/835315641 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdd1c0043e0 con 0x7fdd241029a0 2024-04-25T11:43:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.032+0000 7fdd137fe640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdd0c0a3bb0 0x7fdd0c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.032+0000 7fdd2259c640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdd0c0a3bb0 0x7fdd0c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.032+0000 7fdd137fe640 1 -- 172.21.15.83:0/835315641 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fdd1c0f1d00 con 0x7fdd241029a0 2024-04-25T11:43:31.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.032+0000 7fdd2259c640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdd0c0a3bb0 0x7fdd0c0a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdd14005d00 tx=0x7fdd14005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:31.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.036+0000 7fdd137fe640 1 -- 172.21.15.83:0/835315641 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd1c0be6f0 con 0x7fdd241029a0 2024-04-25T11:43:31.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.216+0000 7fdd28abb640 1 -- 172.21.15.83:0/835315641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdd24062b50 con 0x7fdd241029a0 2024-04-25T11:43:31.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.216+0000 7fdd137fe640 1 -- 172.21.15.83:0/835315641 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fdd1c0de070 con 0x7fdd241029a0 2024-04-25T11:43:31.202 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T11:43:31.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.218+0000 7fdd28abb640 1 -- 172.21.15.83:0/835315641 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdd0c0a3bb0 msgr2=0x7fdd0c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.218+0000 7fdd28abb640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdd0c0a3bb0 0x7fdd0c0a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdd14005d00 tx=0x7fdd14005ed0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.218+0000 7fdd28abb640 1 -- 172.21.15.83:0/835315641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd241029a0 msgr2=0x7fdd2419f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.218+0000 7fdd28abb640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd241029a0 0x7fdd2419f4f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0097b0 tx=0x7fdd1c003c20 comp rx=0 tx=0).stop 2024-04-25T11:43:31.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.218+0000 7fdd28abb640 1 -- 172.21.15.83:0/835315641 shutdown_connections 2024-04-25T11:43:31.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.218+0000 7fdd28abb640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdd0c0a3bb0 0x7fdd0c0a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.218+0000 7fdd28abb640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd241029a0 0x7fdd2419f4f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.218+0000 7fdd28abb640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd24101fe0 0x7fdd2419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.218+0000 7fdd28abb640 1 --2- 172.21.15.83:0/835315641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd24100e90 0x7fdd2406b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.218+0000 7fdd28abb640 1 -- 172.21.15.83:0/835315641 >> 172.21.15.83:0/835315641 conn(0x7fdd240fc440 msgr2=0x7fdd24105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:31.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.218+0000 7fdd28abb640 1 -- 172.21.15.83:0/835315641 shutdown_connections 2024-04-25T11:43:31.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.218+0000 7fdd28abb640 1 -- 172.21.15.83:0/835315641 wait complete. 2024-04-25T11:43:31.214 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:31 smithi083 ceph-mon[29330]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-25T11:43:31.214 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:31 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/780209879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:31.214 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:31 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1401689950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:43:31.214 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:31 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2393390472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:43:31.215 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:43:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.326+0000 7f20e22a3640 1 -- 172.21.15.83:0/4142880666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20dc103190 msgr2=0x7f20dc103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.326+0000 7f20e22a3640 1 --2- 172.21.15.83:0/4142880666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20dc103190 0x7f20dc103610 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009900 tx=0x7f20cc02a410 comp rx=0 tx=0).stop 2024-04-25T11:43:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.326+0000 7f20e22a3640 1 -- 172.21.15.83:0/4142880666 shutdown_connections 2024-04-25T11:43:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.326+0000 7f20e22a3640 1 --2- 172.21.15.83:0/4142880666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20dc103190 0x7f20dc103610 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.326+0000 7f20e22a3640 1 --2- 172.21.15.83:0/4142880666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20dc103b50 0x7f20dc10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.326+0000 7f20e22a3640 1 --2- 172.21.15.83:0/4142880666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20dc101fa0 0x7f20dc1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.326+0000 7f20e22a3640 1 -- 172.21.15.83:0/4142880666 >> 172.21.15.83:0/4142880666 conn(0x7f20dc0fd530 msgr2=0x7f20dc0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:31.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.327+0000 7f20e22a3640 1 -- 172.21.15.83:0/4142880666 shutdown_connections 2024-04-25T11:43:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.327+0000 7f20e22a3640 1 -- 172.21.15.83:0/4142880666 wait complete. 2024-04-25T11:43:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.327+0000 7f20e22a3640 1 Processor -- start 2024-04-25T11:43:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.327+0000 7f20e22a3640 1 -- start start 2024-04-25T11:43:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.327+0000 7f20e22a3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20dc101fa0 0x7f20dc198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.327+0000 7f20e22a3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20dc103190 0x7f20dc198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.327+0000 7f20e22a3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20dc103b50 0x7f20dc19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.327+0000 7f20e22a3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc10db40 con 0x7f20dc101fa0 2024-04-25T11:43:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.327+0000 7f20e22a3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc10d9c0 con 0x7f20dc103190 2024-04-25T11:43:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.327+0000 7f20e22a3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc10dcc0 con 0x7f20dc103b50 2024-04-25T11:43:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.327+0000 7f20dbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20dc101fa0 0x7f20dc198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20dbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20dc101fa0 0x7f20dc198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48674/0 (socket says 172.21.15.83:48674) 2024-04-25T11:43:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20dbfff640 1 -- 172.21.15.83:0/915005318 learned_addr learned my addr 172.21.15.83:0/915005318 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20e0840640 1 --2- 172.21.15.83:0/915005318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20dc103b50 0x7f20dc19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20dbfff640 1 -- 172.21.15.83:0/915005318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20dc103b50 msgr2=0x7f20dc19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20dbfff640 1 --2- 172.21.15.83:0/915005318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20dc103b50 0x7f20dc19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20db7fe640 1 --2- 172.21.15.83:0/915005318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20dc103190 0x7f20dc198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20dbfff640 1 -- 172.21.15.83:0/915005318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20dc103190 msgr2=0x7f20dc198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20dbfff640 1 --2- 172.21.15.83:0/915005318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20dc103190 0x7f20dc198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20dbfff640 1 -- 172.21.15.83:0/915005318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc19fe60 con 0x7f20dc101fa0 2024-04-25T11:43:31.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20e0840640 1 --2- 172.21.15.83:0/915005318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20dc103b50 0x7f20dc19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:31.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20dbfff640 1 --2- 172.21.15.83:0/915005318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20dc101fa0 0x7f20dc198240 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f20d000c970 tx=0x7f20d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:31.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20d97fa640 1 -- 172.21.15.83:0/915005318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0004220 con 0x7f20dc101fa0 2024-04-25T11:43:31.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20d97fa640 1 -- 172.21.15.83:0/915005318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20d00043c0 con 0x7f20dc101fa0 2024-04-25T11:43:31.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20e22a3640 1 -- 172.21.15.83:0/915005318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20dc1a0150 con 0x7f20dc101fa0 2024-04-25T11:43:31.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20d97fa640 1 -- 172.21.15.83:0/915005318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0004a00 con 0x7f20dc101fa0 2024-04-25T11:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.328+0000 7f20e22a3640 1 -- 172.21.15.83:0/915005318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20dc1a06d0 con 0x7f20dc101fa0 2024-04-25T11:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.329+0000 7f20e22a3640 1 -- 172.21.15.83:0/915005318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20dc109390 con 0x7f20dc101fa0 2024-04-25T11:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.330+0000 7f20d97fa640 1 -- 172.21.15.83:0/915005318 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f20d00079b0 con 0x7f20dc101fa0 2024-04-25T11:43:31.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.331+0000 7f20d97fa640 1 --2- 172.21.15.83:0/915005318 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f20c40a3bb0 0x7f20c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.331+0000 7f20db7fe640 1 --2- 172.21.15.83:0/915005318 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f20c40a3bb0 0x7f20c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.331+0000 7f20d97fa640 1 -- 172.21.15.83:0/915005318 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f20d00d0200 con 0x7f20dc101fa0 2024-04-25T11:43:31.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.331+0000 7f20db7fe640 1 --2- 172.21.15.83:0/915005318 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f20c40a3bb0 0x7f20c40a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f20cc02a930 tx=0x7f20cc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:31.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:31 smithi111 ceph-mon[32784]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-25T11:43:31.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:31 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/780209879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:31.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:31 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1401689950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:43:31.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:31 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2393390472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:43:31.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.335+0000 7f20d97fa640 1 -- 172.21.15.83:0/915005318 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20d0014030 con 0x7f20dc101fa0 2024-04-25T11:43:31.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:31 smithi094 ceph-mon[33032]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-25T11:43:31.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:31 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/780209879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:31.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:31 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1401689950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:43:31.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:31 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2393390472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:43:31.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.550+0000 7f20e22a3640 1 -- 172.21.15.83:0/915005318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20dc02d5c0 con 0x7f20dc101fa0 2024-04-25T11:43:31.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.550+0000 7f20d97fa640 1 -- 172.21.15.83:0/915005318 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7f20d009cbf0 con 0x7f20dc101fa0 2024-04-25T11:43:31.537 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:31.537 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:41:44.787926+0000","modified":"2024-04-25T11:41:51.578220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24469,"mds_2":24442,"mds_3":24440,"mds_4":24430},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/1568064835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1568064835},{"type":"v1","addr":"172.21.15.83:6839","nonce":1568064835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/1576544313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1576544313},{"type":"v1","addr":"172.21.15.83:6841","nonce":1576544313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":4,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.94:6837/2475976592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2475976592},{"type":"v1","addr":"172.21.15.94:6837","nonce":2475976592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.111:6837/2927430878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2927430878},{"type":"v1","addr":"172.21.15.111:6837","nonce":2927430878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/2233531151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2233531151},{"type":"v1","addr":"172.21.15.111:6839","nonce":2233531151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/1461078471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1461078471},{"type":"v1","addr":"172.21.15.94:6839","nonce":1461078471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24440,24430,24479,24469,24442]},"id":1}]} 2024-04-25T11:43:31.537 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-04-25T11:43:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.552+0000 7f20e22a3640 1 -- 172.21.15.83:0/915005318 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f20c40a3bb0 msgr2=0x7f20c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.552+0000 7f20e22a3640 1 --2- 172.21.15.83:0/915005318 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f20c40a3bb0 0x7f20c40a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f20cc02a930 tx=0x7f20cc000bc0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.552+0000 7f20e22a3640 1 -- 172.21.15.83:0/915005318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20dc101fa0 msgr2=0x7f20dc198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.552+0000 7f20e22a3640 1 --2- 172.21.15.83:0/915005318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20dc101fa0 0x7f20dc198240 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f20d000c970 tx=0x7f20d000ce30 comp rx=0 tx=0).stop 2024-04-25T11:43:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.552+0000 7f20e22a3640 1 -- 172.21.15.83:0/915005318 shutdown_connections 2024-04-25T11:43:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.552+0000 7f20e22a3640 1 --2- 172.21.15.83:0/915005318 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f20c40a3bb0 0x7f20c40a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.552+0000 7f20e22a3640 1 --2- 172.21.15.83:0/915005318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20dc103b50 0x7f20dc19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.552+0000 7f20e22a3640 1 --2- 172.21.15.83:0/915005318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20dc103190 0x7f20dc198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.552+0000 7f20e22a3640 1 --2- 172.21.15.83:0/915005318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20dc101fa0 0x7f20dc198240 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.552+0000 7f20e22a3640 1 -- 172.21.15.83:0/915005318 >> 172.21.15.83:0/915005318 conn(0x7f20dc0fd530 msgr2=0x7f20dc106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.552+0000 7f20e22a3640 1 -- 172.21.15.83:0/915005318 shutdown_connections 2024-04-25T11:43:31.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.552+0000 7f20e22a3640 1 -- 172.21.15.83:0/915005318 wait complete. 2024-04-25T11:43:31.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:43:31.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.661+0000 7f61f44fa640 1 -- 172.21.15.83:0/268914655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ec102a70 msgr2=0x7f61ec104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.661+0000 7f61f44fa640 1 --2- 172.21.15.83:0/268914655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ec102a70 0x7f61ec104e60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009900 tx=0x7f61e802a410 comp rx=0 tx=0).stop 2024-04-25T11:43:31.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f44fa640 1 -- 172.21.15.83:0/268914655 shutdown_connections 2024-04-25T11:43:31.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f44fa640 1 --2- 172.21.15.83:0/268914655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61ec107cd0 0x7f61ec10a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f44fa640 1 --2- 172.21.15.83:0/268914655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61ec1053a0 0x7f61ec107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f44fa640 1 --2- 172.21.15.83:0/268914655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ec102a70 0x7f61ec104e60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f44fa640 1 -- 172.21.15.83:0/268914655 >> 172.21.15.83:0/268914655 conn(0x7f61ec0fc400 msgr2=0x7f61ec0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f44fa640 1 -- 172.21.15.83:0/268914655 shutdown_connections 2024-04-25T11:43:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f44fa640 1 -- 172.21.15.83:0/268914655 wait complete. 2024-04-25T11:43:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f44fa640 1 Processor -- start 2024-04-25T11:43:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f44fa640 1 -- start start 2024-04-25T11:43:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f44fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ec102a70 0x7f61ec1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f44fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61ec1053a0 0x7f61ec198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f44fa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61ec107cd0 0x7f61ec19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f44fa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec10db40 con 0x7f61ec102a70 2024-04-25T11:43:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f44fa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec10d9c0 con 0x7f61ec1053a0 2024-04-25T11:43:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f44fa640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec10dcc0 con 0x7f61ec107cd0 2024-04-25T11:43:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f2296640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ec102a70 0x7f61ec1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f2296640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ec102a70 0x7f61ec1982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48684/0 (socket says 172.21.15.83:48684) 2024-04-25T11:43:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.662+0000 7f61f2296640 1 -- 172.21.15.83:0/2148792051 learned_addr learned my addr 172.21.15.83:0/2148792051 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.663+0000 7f61f2296640 1 -- 172.21.15.83:0/2148792051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61ec107cd0 msgr2=0x7f61ec19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.663+0000 7f61f2296640 1 --2- 172.21.15.83:0/2148792051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61ec107cd0 0x7f61ec19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.663+0000 7f61f2296640 1 -- 172.21.15.83:0/2148792051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61ec1053a0 msgr2=0x7f61ec198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.663+0000 7f61f2296640 1 --2- 172.21.15.83:0/2148792051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61ec1053a0 0x7f61ec198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.663+0000 7f61f2296640 1 -- 172.21.15.83:0/2148792051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ec19fe90 con 0x7f61ec102a70 2024-04-25T11:43:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.663+0000 7f61f2296640 1 --2- 172.21.15.83:0/2148792051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ec102a70 0x7f61ec1982c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f61e8005940 tx=0x7f61e8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.663+0000 7f61e37fe640 1 -- 172.21.15.83:0/2148792051 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e803f030 con 0x7f61ec102a70 2024-04-25T11:43:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.663+0000 7f61e37fe640 1 -- 172.21.15.83:0/2148792051 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61e8003cb0 con 0x7f61ec102a70 2024-04-25T11:43:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.663+0000 7f61f44fa640 1 -- 172.21.15.83:0/2148792051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ec1a0120 con 0x7f61ec102a70 2024-04-25T11:43:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.663+0000 7f61e37fe640 1 -- 172.21.15.83:0/2148792051 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8035070 con 0x7f61ec102a70 2024-04-25T11:43:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.663+0000 7f61f44fa640 1 -- 172.21.15.83:0/2148792051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ec1a0690 con 0x7f61ec102a70 2024-04-25T11:43:31.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.664+0000 7f61f44fa640 1 -- 172.21.15.83:0/2148792051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61ec0fdef0 con 0x7f61ec102a70 2024-04-25T11:43:31.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.666+0000 7f61e37fe640 1 -- 172.21.15.83:0/2148792051 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f61e8042030 con 0x7f61ec102a70 2024-04-25T11:43:31.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.666+0000 7f61e37fe640 1 --2- 172.21.15.83:0/2148792051 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f61d80a3bb0 0x7f61d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.666+0000 7f61f1a95640 1 --2- 172.21.15.83:0/2148792051 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f61d80a3bb0 0x7f61d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.666+0000 7f61e37fe640 1 -- 172.21.15.83:0/2148792051 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f61e80f0fd0 con 0x7f61ec102a70 2024-04-25T11:43:31.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.667+0000 7f61f1a95640 1 --2- 172.21.15.83:0/2148792051 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f61d80a3bb0 0x7f61d80a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f61dc005fd0 tx=0x7f61dc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:31.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.670+0000 7f61e37fe640 1 -- 172.21.15.83:0/2148792051 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f61e80bda60 con 0x7f61ec102a70 2024-04-25T11:43:31.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.846+0000 7f61f44fa640 1 -- 172.21.15.83:0/2148792051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61ec02d820 con 0x7f61ec102a70 2024-04-25T11:43:31.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.847+0000 7f61e37fe640 1 -- 172.21.15.83:0/2148792051 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22581 (secure 0 0 0) 0x7f61e8003e60 con 0x7f61ec102a70 2024-04-25T11:43:31.834 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:43:31.834 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:42:51.074495+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:41:42.546991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:41:43.764226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:43:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.849+0000 7f61f44fa640 1 -- 172.21.15.83:0/2148792051 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f61d80a3bb0 msgr2=0x7f61d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.849+0000 7f61f44fa640 1 --2- 172.21.15.83:0/2148792051 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f61d80a3bb0 0x7f61d80a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f61dc005fd0 tx=0x7f61dc005850 comp rx=0 tx=0).stop 2024-04-25T11:43:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.849+0000 7f61f44fa640 1 -- 172.21.15.83:0/2148792051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ec102a70 msgr2=0x7f61ec1982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.849+0000 7f61f44fa640 1 --2- 172.21.15.83:0/2148792051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ec102a70 0x7f61ec1982c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f61e8005940 tx=0x7f61e8004430 comp rx=0 tx=0).stop 2024-04-25T11:43:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.849+0000 7f61f44fa640 1 -- 172.21.15.83:0/2148792051 shutdown_connections 2024-04-25T11:43:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.849+0000 7f61f44fa640 1 --2- 172.21.15.83:0/2148792051 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f61d80a3bb0 0x7f61d80a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.849+0000 7f61f44fa640 1 --2- 172.21.15.83:0/2148792051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f61ec107cd0 0x7f61ec19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.849+0000 7f61f44fa640 1 --2- 172.21.15.83:0/2148792051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61ec1053a0 0x7f61ec198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.849+0000 7f61f44fa640 1 --2- 172.21.15.83:0/2148792051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61ec102a70 0x7f61ec1982c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.849+0000 7f61f44fa640 1 -- 172.21.15.83:0/2148792051 >> 172.21.15.83:0/2148792051 conn(0x7f61ec0fc400 msgr2=0x7f61ec103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.850+0000 7f61f44fa640 1 -- 172.21.15.83:0/2148792051 shutdown_connections 2024-04-25T11:43:31.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.850+0000 7f61f44fa640 1 -- 172.21.15.83:0/2148792051 wait complete. 2024-04-25T11:43:31.848 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T11:41:44.787926+0000', 'modified': '2024-04-25T11:41:51.578220+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24479, 'mds_1': 24469, 'mds_2': 24442, 'mds_3': 24440, 'mds_4': 24430}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14631': {'gid': 14631, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6839/1568064835', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 1568064835}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 1568064835}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6841/1576544313', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6840', 'nonce': 1576544313}, {'type': 'v1', 'addr': '172.21.15.83:6841', 'nonce': 1576544313}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24391': {'gid': 24391, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6833/2234401967', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 2234401967}, {'type': 'v1', 'addr': '172.21.15.111:6833', 'nonce': 2234401967}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24392': {'gid': 24392, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6833/3250237836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6832', 'nonce': 3250237836}, {'type': 'v1', 'addr': '172.21.15.94:6833', 'nonce': 3250237836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24416': {'gid': 24416, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6835/4086957377', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6834', 'nonce': 4086957377}, {'type': 'v1', 'addr': '172.21.15.94:6835', 'nonce': 4086957377}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.111:6835/4167388778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 4167388778}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 4167388778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.94:6837/2475976592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6836', 'nonce': 2475976592}, {'type': 'v1', 'addr': '172.21.15.94:6837', 'nonce': 2475976592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.111:6837/2927430878', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 2927430878}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 2927430878}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.111:6839/2233531151', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 2233531151}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 2233531151}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.94:6839/1461078471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6838', 'nonce': 1461078471}, {'type': 'v1', 'addr': '172.21.15.94:6839', 'nonce': 1461078471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24479, 'qdb_cluster': [24440, 24430, 24479, 24469, 24442]} 2024-04-25T11:43:31.848 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-25T11:43:31.848 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-25T11:43:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.959+0000 7faa59ba2640 1 -- 172.21.15.83:0/1134930250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa541029a0 msgr2=0x7faa541092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.959+0000 7faa59ba2640 1 --2- 172.21.15.83:0/1134930250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa541029a0 0x7faa541092c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009ca0 tx=0x7faa4c02a8c0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.959+0000 7faa59ba2640 1 -- 172.21.15.83:0/1134930250 shutdown_connections 2024-04-25T11:43:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.959+0000 7faa59ba2640 1 --2- 172.21.15.83:0/1134930250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa541029a0 0x7faa541092c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.959+0000 7faa59ba2640 1 --2- 172.21.15.83:0/1134930250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa54101fe0 0x7faa54102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.959+0000 7faa59ba2640 1 --2- 172.21.15.83:0/1134930250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa54100e90 0x7faa541012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.959+0000 7faa59ba2640 1 -- 172.21.15.83:0/1134930250 >> 172.21.15.83:0/1134930250 conn(0x7faa540fc400 msgr2=0x7faa540fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.959+0000 7faa59ba2640 1 -- 172.21.15.83:0/1134930250 shutdown_connections 2024-04-25T11:43:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.959+0000 7faa59ba2640 1 -- 172.21.15.83:0/1134930250 wait complete. 2024-04-25T11:43:31.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.960+0000 7faa59ba2640 1 Processor -- start 2024-04-25T11:43:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.960+0000 7faa59ba2640 1 -- start start 2024-04-25T11:43:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.960+0000 7faa59ba2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa54100e90 0x7faa54069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.960+0000 7faa59ba2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa54101fe0 0x7faa5419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.960+0000 7faa59ba2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa541029a0 0x7faa5419f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.960+0000 7faa59ba2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa5410c9f0 con 0x7faa54101fe0 2024-04-25T11:43:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.960+0000 7faa59ba2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa5410c870 con 0x7faa54100e90 2024-04-25T11:43:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.960+0000 7faa59ba2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faa5410cb70 con 0x7faa541029a0 2024-04-25T11:43:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.960+0000 7faa52ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa54101fe0 0x7faa5419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.960+0000 7faa52ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa54101fe0 0x7faa5419cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48698/0 (socket says 172.21.15.83:48698) 2024-04-25T11:43:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.960+0000 7faa52ffd640 1 -- 172.21.15.83:0/1522413370 learned_addr learned my addr 172.21.15.83:0/1522413370 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.961+0000 7faa53fff640 1 --2- 172.21.15.83:0/1522413370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa541029a0 0x7faa5419f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.961+0000 7faa537fe640 1 --2- 172.21.15.83:0/1522413370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa54100e90 0x7faa54069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.961+0000 7faa52ffd640 1 -- 172.21.15.83:0/1522413370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa541029a0 msgr2=0x7faa5419f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.961+0000 7faa52ffd640 1 --2- 172.21.15.83:0/1522413370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa541029a0 0x7faa5419f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.961+0000 7faa52ffd640 1 -- 172.21.15.83:0/1522413370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa54100e90 msgr2=0x7faa54069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.961+0000 7faa52ffd640 1 --2- 172.21.15.83:0/1522413370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa54100e90 0x7faa54069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.961+0000 7faa52ffd640 1 -- 172.21.15.83:0/1522413370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5419fdc0 con 0x7faa54101fe0 2024-04-25T11:43:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.961+0000 7faa537fe640 1 --2- 172.21.15.83:0/1522413370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa54100e90 0x7faa54069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.961+0000 7faa52ffd640 1 --2- 172.21.15.83:0/1522413370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa54101fe0 0x7faa5419cec0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faa4400c970 tx=0x7faa4400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.961+0000 7faa50ff9640 1 -- 172.21.15.83:0/1522413370 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44004220 con 0x7faa54101fe0 2024-04-25T11:43:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.961+0000 7faa59ba2640 1 -- 172.21.15.83:0/1522413370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa541a00b0 con 0x7faa54101fe0 2024-04-25T11:43:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.961+0000 7faa50ff9640 1 -- 172.21.15.83:0/1522413370 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa440043c0 con 0x7faa54101fe0 2024-04-25T11:43:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.961+0000 7faa50ff9640 1 -- 172.21.15.83:0/1522413370 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4400f450 con 0x7faa54101fe0 2024-04-25T11:43:31.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.961+0000 7faa59ba2640 1 -- 172.21.15.83:0/1522413370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa541a0650 con 0x7faa54101fe0 2024-04-25T11:43:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.963+0000 7faa50ff9640 1 -- 172.21.15.83:0/1522413370 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faa44003610 con 0x7faa54101fe0 2024-04-25T11:43:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.963+0000 7faa5c996740 1 Processor -- start 2024-04-25T11:43:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.963+0000 7faa5c996740 1 -- start start 2024-04-25T11:43:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.963+0000 7faa5c996740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59db80 0x55adcb59dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa5c996740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59ed50 0x55adcb59f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa5c996740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa5c996740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55adcb502c50 con 0x55adcb59f710 2024-04-25T11:43:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa5c996740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55adcb49f830 con 0x55adcb59db80 2024-04-25T11:43:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa5c996740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55adcb4fec10 con 0x55adcb59ed50 2024-04-25T11:43:31.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa50ff9640 1 --2- 172.21.15.83:0/1522413370 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faa400a3bb0 0x7faa400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa52ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa52ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5a5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48710/0 (socket says 172.21.15.83:48710) 2024-04-25T11:43:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa52ffd640 1 -- 172.21.15.83:0/347754353 learned_addr learned my addr 172.21.15.83:0/347754353 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa53fff640 1 --2- 172.21.15.83:0/1522413370 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faa400a3bb0 0x7faa400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa53fff640 1 --2- 172.21.15.83:0/347754353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59db80 0x55adcb59dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa537fe640 1 --2- 172.21.15.83:0/347754353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59ed50 0x55adcb59f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa50ff9640 1 -- 172.21.15.83:0/1522413370 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7faa440d0270 con 0x7faa54101fe0 2024-04-25T11:43:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa52ffd640 1 -- 172.21.15.83:0/347754353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59ed50 msgr2=0x55adcb59f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa52ffd640 1 --2- 172.21.15.83:0/347754353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59ed50 0x55adcb59f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa52ffd640 1 -- 172.21.15.83:0/347754353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59db80 msgr2=0x55adcb59dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa52ffd640 1 --2- 172.21.15.83:0/347754353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59db80 0x55adcb59dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa52ffd640 1 -- 172.21.15.83:0/347754353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adcb2152d0 con 0x55adcb59f710 2024-04-25T11:43:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa537fe640 1 --2- 172.21.15.83:0/347754353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59ed50 0x55adcb59f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:31.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa52ffd640 1 --2- 172.21.15.83:0/347754353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5a5fa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faa44016000 tx=0x7faa44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63345e647972648e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa53fff640 1 --2- 172.21.15.83:0/1522413370 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faa400a3bb0 0x7faa400a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faa4c031b50 tx=0x7faa4c009890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa49ffb640 1 -- 172.21.15.83:0/347754353 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4401d070 con 0x55adcb59f710 2024-04-25T11:43:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa49ffb640 1 -- 172.21.15.83:0/347754353 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa4409cc60 con 0x55adcb59f710 2024-04-25T11:43:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.964+0000 7faa49ffb640 1 -- 172.21.15.83:0/347754353 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4409a860 con 0x55adcb59f710 2024-04-25T11:43:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.965+0000 7faa5c996740 1 -- 172.21.15.83:0/347754353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 msgr2=0x55adcb5a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.965+0000 7faa5c996740 1 --2- 172.21.15.83:0/347754353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5a5fa0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faa44016000 tx=0x7faa44000bc0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.965+0000 7faa5c996740 1 -- 172.21.15.83:0/347754353 shutdown_connections 2024-04-25T11:43:31.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.965+0000 7faa5c996740 1 --2- 172.21.15.83:0/347754353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5a5fa0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.965+0000 7faa5c996740 1 --2- 172.21.15.83:0/347754353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59ed50 0x55adcb59f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.965+0000 7faa5c996740 1 --2- 172.21.15.83:0/347754353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59db80 0x55adcb59dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.965+0000 7faa5c996740 1 -- 172.21.15.83:0/347754353 >> 172.21.15.83:0/347754353 conn(0x55adcb599070 msgr2=0x55adcb599470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.965+0000 7faa5c996740 1 -- 172.21.15.83:0/347754353 shutdown_connections 2024-04-25T11:43:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.965+0000 7faa5c996740 1 -- 172.21.15.83:0/347754353 wait complete. 2024-04-25T11:43:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.965+0000 7faa5c996740 1 -- start start 2024-04-25T11:43:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa5c996740 1 Processor -- start 2024-04-25T11:43:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa5c996740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59db80 0x55adcb5a8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa5c996740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59ed50 0x55adcb5a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa5c996740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa5c996740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55adcb5a6720 con 0x55adcb59f710 2024-04-25T11:43:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa5c996740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55adcb49d470 con 0x55adcb59db80 2024-04-25T11:43:31.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa5c996740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55adcb5a68a0 con 0x55adcb59ed50 2024-04-25T11:43:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa5c996740 10 client.0 _ll_register_callbacks cb 0x55adcb4c2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa53fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa53fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5af5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48720/0 (socket says 172.21.15.83:48720) 2024-04-25T11:43:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa53fff640 1 -- 172.21.15.83:0/2979068990 learned_addr learned my addr 172.21.15.83:0/2979068990 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa537fe640 1 --2- 172.21.15.83:0/2979068990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59db80 0x55adcb5a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa52ffd640 1 --2- 172.21.15.83:0/2979068990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59ed50 0x55adcb5a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa53fff640 1 -- 172.21.15.83:0/2979068990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59ed50 msgr2=0x55adcb5a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa53fff640 1 --2- 172.21.15.83:0/2979068990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59ed50 0x55adcb5a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa53fff640 1 -- 172.21.15.83:0/2979068990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59db80 msgr2=0x55adcb5a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa53fff640 1 --2- 172.21.15.83:0/2979068990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59db80 0x55adcb5a8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa53fff640 1 -- 172.21.15.83:0/2979068990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adcb2df6b0 con 0x55adcb59f710 2024-04-25T11:43:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa52ffd640 1 --2- 172.21.15.83:0/2979068990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59ed50 0x55adcb5a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa53fff640 1 --2- 172.21.15.83:0/2979068990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5af5f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faa4c033040 tx=0x7faa4c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f9fcafba9229a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:31.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa32ffd640 1 -- 172.21.15.83:0/2979068990 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c050070 con 0x55adcb59f710 2024-04-25T11:43:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa32ffd640 1 -- 172.21.15.83:0/2979068990 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7faa4c0046a0 con 0x55adcb59f710 2024-04-25T11:43:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.966+0000 7faa5c996740 1 -- 172.21.15.83:0/2979068990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa4409cc60 con 0x55adcb59f710 2024-04-25T11:43:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.967+0000 7faa32ffd640 1 -- 172.21.15.83:0/2979068990 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faa4c02f660 con 0x55adcb59f710 2024-04-25T11:43:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.967+0000 7faa5c996740 10 client.15258 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.967+0000 7faa5c996740 1 -- 172.21.15.83:0/2979068990 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55adcb5b2960 con 0x55adcb59f710 2024-04-25T11:43:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.967+0000 7faa32ffd640 1 -- 172.21.15.83:0/2979068990 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa4409cc60 con 0x55adcb59f710 2024-04-25T11:43:31.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.967+0000 7faa32ffd640 1 -- 172.21.15.83:0/2979068990 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7faa4c030730 con 0x55adcb59f710 2024-04-25T11:43:31.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.967+0000 7faa5c996740 10 client.15258 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:31.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.967+0000 7faa5c996740 10 client.15258 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T11:43:31.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.967+0000 7faa5c996740 20 client.15258 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:31.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.967+0000 7faa5c996740 1 --2- 172.21.15.83:0/2979068990 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x55adcb5b5030 0x55adcb5b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.967+0000 7faa5c996740 4 client.15258 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:43:31.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.967+0000 7faa5c996740 1 -- 172.21.15.83:0/2979068990 --> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adcb510670 con 0x55adcb5b5030 2024-04-25T11:43:31.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.967+0000 7faa537fe640 1 --2- 172.21.15.83:0/2979068990 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x55adcb5b5030 0x55adcb5b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.968+0000 7faa32ffd640 10 client.15258 ms_handle_connect on v2:172.21.15.111:6834/4167388778 2024-04-25T11:43:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.968+0000 7faa537fe640 1 --2- 172.21.15.83:0/2979068990 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x55adcb5b5030 0x55adcb5b64a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=743aabfb9efa2f5 server_cookie=4f0629fdff467514 in_seq=0 out_seq=0 2024-04-25T11:43:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.968+0000 7faa32ffd640 1 -- 172.21.15.83:0/2979068990 <== mds.4 v2:172.21.15.111:6834/4167388778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55adcb510670 con 0x55adcb5b5030 2024-04-25T11:43:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.968+0000 7faa32ffd640 10 client.15258 handle_command_reply: tid=0 2024-04-25T11:43:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.968+0000 7faa5c996740 1 client.15258 shutdown 2024-04-25T11:43:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.969+0000 7faa5c996740 1 -- 172.21.15.83:0/2979068990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 msgr2=0x55adcb5af5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.969+0000 7faa5c996740 1 --2- 172.21.15.83:0/2979068990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5af5f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faa4c033040 tx=0x7faa4c009360 comp rx=0 tx=0).stop 2024-04-25T11:43:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.969+0000 7faa5c996740 1 -- 172.21.15.83:0/2979068990 shutdown_connections 2024-04-25T11:43:31.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.969+0000 7faa5c996740 1 --2- 172.21.15.83:0/2979068990 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x55adcb5b5030 0x55adcb5b64a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.969+0000 7faa5c996740 1 --2- 172.21.15.83:0/2979068990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5af5f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.969+0000 7faa5c996740 1 --2- 172.21.15.83:0/2979068990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59ed50 0x55adcb5a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.969+0000 7faa32ffd640 0 client.15258 ms_handle_reset on v2:172.21.15.111:6834/4167388778 2024-04-25T11:43:31.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.969+0000 7faa5c996740 1 --2- 172.21.15.83:0/2979068990 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59db80 0x55adcb5a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.969+0000 7faa5c996740 1 -- 172.21.15.83:0/2979068990 >> 172.21.15.83:0/2979068990 conn(0x55adcb599070 msgr2=0x55adcb5981d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:31.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.969+0000 7faa5c996740 1 -- 172.21.15.83:0/2979068990 shutdown_connections 2024-04-25T11:43:31.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.969+0000 7faa5c996740 1 -- 172.21.15.83:0/2979068990 wait complete. 2024-04-25T11:43:31.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.969+0000 7faa5c996740 20 client.15258 trim_cache size 0 max 16384 2024-04-25T11:43:31.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa5c996740 1 Processor -- start 2024-04-25T11:43:31.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa5c996740 1 -- start start 2024-04-25T11:43:31.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa5c996740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59db80 0x55adcb5a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa5c996740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59ed50 0x55adcb5af510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa5c996740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa5c996740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55adcb5b47e0 con 0x55adcb59f710 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa5c996740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55adcb5b4eb0 con 0x55adcb59ed50 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa5c996740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55adcb5b5580 con 0x55adcb59db80 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa53fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa53fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5940b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48726/0 (socket says 172.21.15.83:48726) 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa53fff640 1 -- 172.21.15.83:0/2116000778 learned_addr learned my addr 172.21.15.83:0/2116000778 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa53fff640 1 -- 172.21.15.83:0/2116000778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59db80 msgr2=0x55adcb5a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa53fff640 1 --2- 172.21.15.83:0/2116000778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59db80 0x55adcb5a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa53fff640 1 -- 172.21.15.83:0/2116000778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59ed50 msgr2=0x55adcb5af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa53fff640 1 --2- 172.21.15.83:0/2116000778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59ed50 0x55adcb5af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa53fff640 1 -- 172.21.15.83:0/2116000778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adcb59e7d0 con 0x55adcb59f710 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa53fff640 1 --2- 172.21.15.83:0/2116000778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5940b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009a60 tx=0x7faa4c032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2681baa784396b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa497fa640 1 -- 172.21.15.83:0/2116000778 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c050040 con 0x55adcb59f710 2024-04-25T11:43:31.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa497fa640 1 -- 172.21.15.83:0/2116000778 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa4c003c80 con 0x55adcb59f710 2024-04-25T11:43:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.987+0000 7faa497fa640 1 -- 172.21.15.83:0/2116000778 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c02f930 con 0x55adcb59f710 2024-04-25T11:43:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.988+0000 7faa5c996740 1 -- 172.21.15.83:0/2116000778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 msgr2=0x55adcb5940b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.988+0000 7faa5c996740 1 --2- 172.21.15.83:0/2116000778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5940b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009a60 tx=0x7faa4c032eb0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.988+0000 7faa5c996740 1 -- 172.21.15.83:0/2116000778 shutdown_connections 2024-04-25T11:43:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.988+0000 7faa5c996740 1 --2- 172.21.15.83:0/2116000778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59f710 0x55adcb5940b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.988+0000 7faa5c996740 1 --2- 172.21.15.83:0/2116000778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59ed50 0x55adcb5af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.988+0000 7faa5c996740 1 --2- 172.21.15.83:0/2116000778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59db80 0x55adcb5a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.988+0000 7faa5c996740 1 -- 172.21.15.83:0/2116000778 >> 172.21.15.83:0/2116000778 conn(0x55adcb599070 msgr2=0x55adcb599450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.988+0000 7faa5c996740 1 -- 172.21.15.83:0/2116000778 shutdown_connections 2024-04-25T11:43:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.988+0000 7faa5c996740 1 -- 172.21.15.83:0/2116000778 wait complete. 2024-04-25T11:43:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.988+0000 7faa5c996740 1 -- start start 2024-04-25T11:43:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.988+0000 7faa5c996740 1 Processor -- start 2024-04-25T11:43:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa5c996740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59db80 0x55adcb5ab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa5c996740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59ed50 0x55adcb5ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa5c996740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59f710 0x55adcb5adf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa5c996740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55adcb5b6320 con 0x55adcb59ed50 2024-04-25T11:43:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa5c996740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55adcb5b5c50 con 0x55adcb59f710 2024-04-25T11:43:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa5c996740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55adcb5b69f0 con 0x55adcb59db80 2024-04-25T11:43:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa5c996740 10 client.0 _ll_register_callbacks cb 0x7faa4c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa52ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59ed50 0x55adcb5ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa52ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59ed50 0x55adcb5ada40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48740/0 (socket says 172.21.15.83:48740) 2024-04-25T11:43:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa52ffd640 1 -- 172.21.15.83:0/674250674 learned_addr learned my addr 172.21.15.83:0/674250674 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa52ffd640 1 -- 172.21.15.83:0/674250674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59db80 msgr2=0x55adcb5ab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa52ffd640 1 --2- 172.21.15.83:0/674250674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59db80 0x55adcb5ab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa52ffd640 1 -- 172.21.15.83:0/674250674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59f710 msgr2=0x55adcb5adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa52ffd640 1 --2- 172.21.15.83:0/674250674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59f710 0x55adcb5adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa52ffd640 1 -- 172.21.15.83:0/674250674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adcb5ae7b0 con 0x55adcb59ed50 2024-04-25T11:43:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa52ffd640 1 --2- 172.21.15.83:0/674250674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59ed50 0x55adcb5ada40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7faa4409c260 tx=0x7faa4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9703b1be8b42bae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa32ffd640 1 -- 172.21.15.83:0/674250674 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa440b5060 con 0x55adcb59ed50 2024-04-25T11:43:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa32ffd640 1 -- 172.21.15.83:0/674250674 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7faa4409c7c0 con 0x55adcb59ed50 2024-04-25T11:43:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa5c996740 1 -- 172.21.15.83:0/674250674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa4c003c80 con 0x55adcb59ed50 2024-04-25T11:43:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa32ffd640 1 -- 172.21.15.83:0/674250674 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faa4401d040 con 0x55adcb59ed50 2024-04-25T11:43:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa5c996740 10 client.15264 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa5c996740 1 -- 172.21.15.83:0/674250674 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55adcb61afe0 con 0x55adcb59ed50 2024-04-25T11:43:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.989+0000 7faa32ffd640 1 -- 172.21.15.83:0/674250674 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa4c003c80 con 0x55adcb59ed50 2024-04-25T11:43:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.990+0000 7faa32ffd640 1 -- 172.21.15.83:0/674250674 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7faa440b5ce0 con 0x55adcb59ed50 2024-04-25T11:43:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.990+0000 7faa5c996740 10 client.15264 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:31.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.990+0000 7faa5c996740 10 client.15264 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-25T11:43:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.990+0000 7faa5c996740 20 client.15264 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.990+0000 7faa5c996740 1 --2- 172.21.15.83:0/674250674 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x55adcb61f590 0x55adcb61e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.990+0000 7faa5c996740 4 client.15264 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:43:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.990+0000 7faa5c996740 1 -- 172.21.15.83:0/674250674 --> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] -- command(tid 0: {"prefix": "status"}) -- 0x55adcb50a4e0 con 0x55adcb61f590 2024-04-25T11:43:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.990+0000 7faa537fe640 1 --2- 172.21.15.83:0/674250674 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x55adcb61f590 0x55adcb61e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.990+0000 7faa32ffd640 10 client.15264 ms_handle_connect on v2:172.21.15.111:6834/4167388778 2024-04-25T11:43:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.990+0000 7faa537fe640 1 --2- 172.21.15.83:0/674250674 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x55adcb61f590 0x55adcb61e2a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f3d1e955e5184118 server_cookie=e2279effa5d46e78 in_seq=0 out_seq=0 2024-04-25T11:43:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa32ffd640 1 -- 172.21.15.83:0/674250674 <== mds.4 v2:172.21.15.111:6834/4167388778 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55adcb50a4e0 con 0x55adcb61f590 2024-04-25T11:43:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa32ffd640 10 client.15264 handle_command_reply: tid=0 2024-04-25T11:43:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa5c996740 1 client.15264 shutdown 2024-04-25T11:43:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa5c996740 1 -- 172.21.15.83:0/674250674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59ed50 msgr2=0x55adcb5ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa5c996740 1 --2- 172.21.15.83:0/674250674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59ed50 0x55adcb5ada40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7faa4409c260 tx=0x7faa4400c8c0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa5c996740 1 -- 172.21.15.83:0/674250674 shutdown_connections 2024-04-25T11:43:31.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa5c996740 1 --2- 172.21.15.83:0/674250674 >> [v2:172.21.15.111:6834/4167388778,v1:172.21.15.111:6835/4167388778] conn(0x55adcb61f590 0x55adcb61e2a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa5c996740 1 --2- 172.21.15.83:0/674250674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55adcb59ed50 0x55adcb5ada40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa5c996740 1 --2- 172.21.15.83:0/674250674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adcb59f710 0x55adcb5adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa32ffd640 0 client.15264 ms_handle_reset on v2:172.21.15.111:6834/4167388778 2024-04-25T11:43:31.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa5c996740 1 --2- 172.21.15.83:0/674250674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55adcb59db80 0x55adcb5ab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa5c996740 1 -- 172.21.15.83:0/674250674 >> 172.21.15.83:0/674250674 conn(0x55adcb599070 msgr2=0x55adcb5940b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:31.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa5c996740 1 -- 172.21.15.83:0/674250674 shutdown_connections 2024-04-25T11:43:31.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa5c996740 1 -- 172.21.15.83:0/674250674 wait complete. 2024-04-25T11:43:31.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa5c996740 20 client.15264 trim_cache size 0 max 16384 2024-04-25T11:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "7fcf54c6-02f7-11ef-bc93-c7b262605968", 2024-04-25T11:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 4, 2024-04-25T11:43:31.984 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24430, 2024-04-25T11:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T11:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T11:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T11:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 101.37621679199999, 2024-04-25T11:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-04-25T11:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-25T11:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T11:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 128.573802693 2024-04-25T11:43:31.985 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:43:31.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa59ba2640 1 -- 172.21.15.83:0/1522413370 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faa400a3bb0 msgr2=0x7faa400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa59ba2640 1 --2- 172.21.15.83:0/1522413370 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faa400a3bb0 0x7faa400a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faa4c031b50 tx=0x7faa4c009890 comp rx=0 tx=0).stop 2024-04-25T11:43:31.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa59ba2640 1 -- 172.21.15.83:0/1522413370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa54101fe0 msgr2=0x7faa5419cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:31.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.991+0000 7faa59ba2640 1 --2- 172.21.15.83:0/1522413370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa54101fe0 0x7faa5419cec0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faa4400c970 tx=0x7faa4400ce30 comp rx=0 tx=0).stop 2024-04-25T11:43:31.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.992+0000 7faa59ba2640 1 -- 172.21.15.83:0/1522413370 shutdown_connections 2024-04-25T11:43:31.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.992+0000 7faa59ba2640 1 --2- 172.21.15.83:0/1522413370 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faa400a3bb0 0x7faa400a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.992+0000 7faa59ba2640 1 --2- 172.21.15.83:0/1522413370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa54101fe0 0x7faa5419cec0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.992+0000 7faa59ba2640 1 --2- 172.21.15.83:0/1522413370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa541029a0 0x7faa5419f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.992+0000 7faa59ba2640 1 --2- 172.21.15.83:0/1522413370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa54100e90 0x7faa54069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:31.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.992+0000 7faa59ba2640 1 -- 172.21.15.83:0/1522413370 >> 172.21.15.83:0/1522413370 conn(0x7faa540fc400 msgr2=0x7faa54105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:31.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.992+0000 7faa59ba2640 1 -- 172.21.15.83:0/1522413370 shutdown_connections 2024-04-25T11:43:31.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:31.992+0000 7faa59ba2640 1 -- 172.21.15.83:0/1522413370 wait complete. 2024-04-25T11:43:31.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T11:43:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.107+0000 7faa70b44640 1 -- 172.21.15.83:0/586113588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c102a20 msgr2=0x7faa6c109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.107+0000 7faa70b44640 1 --2- 172.21.15.83:0/586113588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c102a20 0x7faa6c109340 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faa64009ca0 tx=0x7faa6402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.107+0000 7faa70b44640 1 -- 172.21.15.83:0/586113588 shutdown_connections 2024-04-25T11:43:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.107+0000 7faa70b44640 1 --2- 172.21.15.83:0/586113588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c102a20 0x7faa6c109340 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.107+0000 7faa70b44640 1 --2- 172.21.15.83:0/586113588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c102060 0x7faa6c1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.107+0000 7faa70b44640 1 --2- 172.21.15.83:0/586113588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6c100e80 0x7faa6c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.107+0000 7faa70b44640 1 -- 172.21.15.83:0/586113588 >> 172.21.15.83:0/586113588 conn(0x7faa6c0fc440 msgr2=0x7faa6c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.107+0000 7faa70b44640 1 -- 172.21.15.83:0/586113588 shutdown_connections 2024-04-25T11:43:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.107+0000 7faa70b44640 1 -- 172.21.15.83:0/586113588 wait complete. 2024-04-25T11:43:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.108+0000 7faa70b44640 1 Processor -- start 2024-04-25T11:43:32.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.108+0000 7faa70b44640 1 -- start start 2024-04-25T11:43:32.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.108+0000 7faa70b44640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c100e80 0x7faa6c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.108+0000 7faa70b44640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6c102060 0x7faa6c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.108+0000 7faa70b44640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c102a20 0x7faa6c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.108+0000 7faa70b44640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c10caa0 con 0x7faa6c100e80 2024-04-25T11:43:32.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.108+0000 7faa70b44640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c10c920 con 0x7faa6c102a20 2024-04-25T11:43:32.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.108+0000 7faa70b44640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c10cc20 con 0x7faa6c102060 2024-04-25T11:43:32.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.108+0000 7faa6a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c100e80 0x7faa6c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.108+0000 7faa6a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c100e80 0x7faa6c198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48768/0 (socket says 172.21.15.83:48768) 2024-04-25T11:43:32.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.108+0000 7faa6a59c640 1 -- 172.21.15.83:0/4230532117 learned_addr learned my addr 172.21.15.83:0/4230532117 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.108+0000 7faa69d9b640 1 --2- 172.21.15.83:0/4230532117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6c102060 0x7faa6c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.108+0000 7faa6ad9d640 1 --2- 172.21.15.83:0/4230532117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c102a20 0x7faa6c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.108+0000 7faa6a59c640 1 -- 172.21.15.83:0/4230532117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6c102060 msgr2=0x7faa6c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.109+0000 7faa6a59c640 1 --2- 172.21.15.83:0/4230532117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6c102060 0x7faa6c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.109+0000 7faa6a59c640 1 -- 172.21.15.83:0/4230532117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c102a20 msgr2=0x7faa6c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.109+0000 7faa6a59c640 1 --2- 172.21.15.83:0/4230532117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c102a20 0x7faa6c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.109+0000 7faa6a59c640 1 -- 172.21.15.83:0/4230532117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c19fe50 con 0x7faa6c100e80 2024-04-25T11:43:32.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.109+0000 7faa6ad9d640 1 --2- 172.21.15.83:0/4230532117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c102a20 0x7faa6c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:32.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.109+0000 7faa6a59c640 1 --2- 172.21.15.83:0/4230532117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c100e80 0x7faa6c198230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00c970 tx=0x7faa5c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.109+0000 7faa5b7fe640 1 -- 172.21.15.83:0/4230532117 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c004220 con 0x7faa6c100e80 2024-04-25T11:43:32.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.109+0000 7faa70b44640 1 -- 172.21.15.83:0/4230532117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c1a0140 con 0x7faa6c100e80 2024-04-25T11:43:32.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.109+0000 7faa5b7fe640 1 -- 172.21.15.83:0/4230532117 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa5c0043c0 con 0x7faa6c100e80 2024-04-25T11:43:32.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.109+0000 7faa5b7fe640 1 -- 172.21.15.83:0/4230532117 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c0047c0 con 0x7faa6c100e80 2024-04-25T11:43:32.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.109+0000 7faa70b44640 1 -- 172.21.15.83:0/4230532117 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c1a0710 con 0x7faa6c100e80 2024-04-25T11:43:32.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.111+0000 7faa5b7fe640 1 -- 172.21.15.83:0/4230532117 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faa5c0079b0 con 0x7faa6c100e80 2024-04-25T11:43:32.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.111+0000 7faa73868740 1 Processor -- start 2024-04-25T11:43:32.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa73868740 1 -- start start 2024-04-25T11:43:32.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa73868740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3477f90 0x5612c34783b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa73868740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3479160 0x5612c34795e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa73868740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c34803b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa73868740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5612c33d8c50 con 0x5612c3479b20 2024-04-25T11:43:32.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa73868740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5612c3375830 con 0x5612c3479160 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa73868740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5612c33d4c10 con 0x5612c3477f90 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa69d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c34803b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa6ad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3477f90 0x5612c34783b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa69d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c34803b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48780/0 (socket says 172.21.15.83:48780) 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa69d9b640 1 -- 172.21.15.83:0/3590321835 learned_addr learned my addr 172.21.15.83:0/3590321835 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa6a59c640 1 --2- 172.21.15.83:0/3590321835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3479160 0x5612c34795e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa5b7fe640 1 --2- 172.21.15.83:0/4230532117 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faa540a3bb0 0x7faa540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa6ad9d640 1 -- 172.21.15.83:0/3590321835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3479160 msgr2=0x5612c34795e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa6ad9d640 1 --2- 172.21.15.83:0/3590321835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3479160 0x5612c34795e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa6ad9d640 1 -- 172.21.15.83:0/3590321835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 msgr2=0x5612c34803b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa6ad9d640 1 --2- 172.21.15.83:0/3590321835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c34803b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa6ad9d640 1 -- 172.21.15.83:0/3590321835 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612c30eb2d0 con 0x5612c3477f90 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa69d9b640 1 --2- 172.21.15.83:0/3590321835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c34803b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:43:32.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa6a59c640 1 --2- 172.21.15.83:0/3590321835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3479160 0x5612c34795e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa6ad9d640 1 --2- 172.21.15.83:0/4230532117 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faa540a3bb0 0x7faa540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa5b7fe640 1 -- 172.21.15.83:0/4230532117 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7faa5c0d0140 con 0x7faa6c100e80 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.112+0000 7faa6ad9d640 1 --2- 172.21.15.83:0/3590321835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3477f90 0x5612c34783b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faa64005cc0 tx=0x7faa64031450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c85a3509133a4d9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.113+0000 7faa6ad9d640 1 --2- 172.21.15.83:0/4230532117 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faa540a3bb0 0x7faa540a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faa6404b000 tx=0x7faa64048000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.113+0000 7faa58ff9640 1 -- 172.21.15.83:0/3590321835 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa64004510 con 0x5612c3477f90 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.113+0000 7faa58ff9640 1 -- 172.21.15.83:0/3590321835 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa64036070 con 0x5612c3477f90 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.113+0000 7faa58ff9640 1 -- 172.21.15.83:0/3590321835 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa640305c0 con 0x5612c3477f90 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.113+0000 7faa73868740 1 -- 172.21.15.83:0/3590321835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3477f90 msgr2=0x5612c34783b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.113+0000 7faa73868740 1 --2- 172.21.15.83:0/3590321835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3477f90 0x5612c34783b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faa64005cc0 tx=0x7faa64031450 comp rx=0 tx=0).stop 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.113+0000 7faa73868740 1 -- 172.21.15.83:0/3590321835 shutdown_connections 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.113+0000 7faa73868740 1 --2- 172.21.15.83:0/3590321835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c34803b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.113+0000 7faa73868740 1 --2- 172.21.15.83:0/3590321835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3479160 0x5612c34795e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.113+0000 7faa73868740 1 --2- 172.21.15.83:0/3590321835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3477f90 0x5612c34783b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.113+0000 7faa73868740 1 -- 172.21.15.83:0/3590321835 >> 172.21.15.83:0/3590321835 conn(0x5612c346f070 msgr2=0x5612c346f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa73868740 1 -- 172.21.15.83:0/3590321835 shutdown_connections 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa73868740 1 -- 172.21.15.83:0/3590321835 wait complete. 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa73868740 1 -- start start 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa73868740 1 Processor -- start 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa73868740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3477f90 0x5612c34822d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa73868740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3479160 0x5612c3482810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa73868740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c3489850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa73868740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5612c34809b0 con 0x5612c3479b20 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa73868740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5612c3373470 con 0x5612c3479160 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa73868740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5612c3480b30 con 0x5612c3477f90 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa73868740 10 client.0 _ll_register_callbacks cb 0x5612c3398f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa6ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c3489850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa6ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c3489850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48796/0 (socket says 172.21.15.83:48796) 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa6ad9d640 1 -- 172.21.15.83:0/1352486639 learned_addr learned my addr 172.21.15.83:0/1352486639 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa6a59c640 1 --2- 172.21.15.83:0/1352486639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3477f90 0x5612c34822d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.114+0000 7faa69d9b640 1 --2- 172.21.15.83:0/1352486639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3479160 0x5612c3482810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa6ad9d640 1 -- 172.21.15.83:0/1352486639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3477f90 msgr2=0x5612c34822d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa6ad9d640 1 --2- 172.21.15.83:0/1352486639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3477f90 0x5612c34822d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa6ad9d640 1 -- 172.21.15.83:0/1352486639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3479160 msgr2=0x5612c3482810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa6ad9d640 1 --2- 172.21.15.83:0/1352486639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3479160 0x5612c3482810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa6ad9d640 1 -- 172.21.15.83:0/1352486639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612c31b56b0 con 0x5612c3479b20 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa6ad9d640 1 --2- 172.21.15.83:0/1352486639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c3489850 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7faa6402c010 tx=0x7faa64005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9d538511395f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa51ffb640 1 -- 172.21.15.83:0/1352486639 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa64059030 con 0x5612c3479b20 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa51ffb640 1 -- 172.21.15.83:0/1352486639 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7faa6404d040 con 0x5612c3479b20 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa73868740 1 -- 172.21.15.83:0/1352486639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa64036070 con 0x5612c3479b20 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa51ffb640 1 -- 172.21.15.83:0/1352486639 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faa640305c0 con 0x5612c3479b20 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa73868740 10 client.15276 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:32.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa73868740 1 -- 172.21.15.83:0/1352486639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5612c34809b0 con 0x5612c3479b20 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa51ffb640 1 -- 172.21.15.83:0/1352486639 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa64036070 con 0x5612c3479b20 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa51ffb640 1 -- 172.21.15.83:0/1352486639 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7faa640515f0 con 0x5612c3479b20 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.115+0000 7faa73868740 10 client.15276 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.116+0000 7faa73868740 10 client.15276 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.116+0000 7faa73868740 20 client.15276 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.116+0000 7faa73868740 1 --2- 172.21.15.83:0/1352486639 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5612c348eee0 0x5612c3490350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.116+0000 7faa73868740 4 client.15276 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.116+0000 7faa73868740 1 -- 172.21.15.83:0/1352486639 --> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612c33e6670 con 0x5612c348eee0 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.116+0000 7faa6a59c640 1 --2- 172.21.15.83:0/1352486639 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5612c348eee0 0x5612c3490350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.116+0000 7faa51ffb640 10 client.15276 ms_handle_connect on v2:172.21.15.94:6836/2475976592 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.116+0000 7faa6a59c640 1 --2- 172.21.15.83:0/1352486639 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5612c348eee0 0x5612c3490350 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=489c72b98bef6733 server_cookie=2a3d446f8fbb855b in_seq=0 out_seq=0 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.117+0000 7faa51ffb640 1 -- 172.21.15.83:0/1352486639 <== mds.3 v2:172.21.15.94:6836/2475976592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612c33e6670 con 0x5612c348eee0 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.117+0000 7faa51ffb640 10 client.15276 handle_command_reply: tid=0 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.117+0000 7faa73868740 1 client.15276 shutdown 2024-04-25T11:43:32.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.117+0000 7faa73868740 1 -- 172.21.15.83:0/1352486639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 msgr2=0x5612c3489850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.117+0000 7faa73868740 1 --2- 172.21.15.83:0/1352486639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c3489850 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7faa6402c010 tx=0x7faa64005dc0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.117+0000 7faa73868740 1 -- 172.21.15.83:0/1352486639 shutdown_connections 2024-04-25T11:43:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.117+0000 7faa73868740 1 --2- 172.21.15.83:0/1352486639 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5612c348eee0 0x5612c3490350 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.117+0000 7faa73868740 1 --2- 172.21.15.83:0/1352486639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c3489850 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.117+0000 7faa73868740 1 --2- 172.21.15.83:0/1352486639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3479160 0x5612c3482810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.117+0000 7faa51ffb640 0 client.15276 ms_handle_reset on v2:172.21.15.94:6836/2475976592 2024-04-25T11:43:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.117+0000 7faa73868740 1 --2- 172.21.15.83:0/1352486639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3477f90 0x5612c34822d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.117+0000 7faa73868740 1 -- 172.21.15.83:0/1352486639 >> 172.21.15.83:0/1352486639 conn(0x5612c346f070 msgr2=0x5612c346e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.117+0000 7faa73868740 1 -- 172.21.15.83:0/1352486639 shutdown_connections 2024-04-25T11:43:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.117+0000 7faa73868740 1 -- 172.21.15.83:0/1352486639 wait complete. 2024-04-25T11:43:32.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.117+0000 7faa73868740 20 client.15276 trim_cache size 0 max 16384 2024-04-25T11:43:32.116 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:32 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/835315641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:43:32.117 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:32 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/915005318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:43:32.117 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:32 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2148792051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:32.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.136+0000 7faa73868740 1 Processor -- start 2024-04-25T11:43:32.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.136+0000 7faa73868740 1 -- start start 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.136+0000 7faa73868740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3477f90 0x5612c3480ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.136+0000 7faa73868740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479160 0x5612c3489810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.136+0000 7faa73868740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3479b20 0x5612c34822d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.136+0000 7faa73868740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5612c348e690 con 0x5612c3479160 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.136+0000 7faa73868740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5612c348ed60 con 0x5612c3477f90 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.136+0000 7faa73868740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5612c348f430 con 0x5612c3479b20 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.137+0000 7faa69d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479160 0x5612c3489810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.137+0000 7faa69d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479160 0x5612c3489810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48810/0 (socket says 172.21.15.83:48810) 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.137+0000 7faa69d9b640 1 -- 172.21.15.83:0/2394147807 learned_addr learned my addr 172.21.15.83:0/2394147807 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.137+0000 7faa6a59c640 1 --2- 172.21.15.83:0/2394147807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3477f90 0x5612c3480ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.137+0000 7faa6ad9d640 1 --2- 172.21.15.83:0/2394147807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3479b20 0x5612c34822d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.137+0000 7faa69d9b640 1 -- 172.21.15.83:0/2394147807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3479b20 msgr2=0x5612c34822d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.137+0000 7faa69d9b640 1 --2- 172.21.15.83:0/2394147807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3479b20 0x5612c34822d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.137+0000 7faa69d9b640 1 -- 172.21.15.83:0/2394147807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3477f90 msgr2=0x5612c3480ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.137+0000 7faa69d9b640 1 --2- 172.21.15.83:0/2394147807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3477f90 0x5612c3480ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.137+0000 7faa69d9b640 1 -- 172.21.15.83:0/2394147807 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612c346fd40 con 0x5612c3479160 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.137+0000 7faa69d9b640 1 --2- 172.21.15.83:0/2394147807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479160 0x5612c3489810 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faa60009ab0 tx=0x7faa60009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dae1b038c28f662 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.137+0000 7faa53fff640 1 -- 172.21.15.83:0/2394147807 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60022070 con 0x5612c3479160 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.137+0000 7faa53fff640 1 -- 172.21.15.83:0/2394147807 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa60014ca0 con 0x5612c3479160 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.137+0000 7faa53fff640 1 -- 172.21.15.83:0/2394147807 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60015790 con 0x5612c3479160 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.138+0000 7faa73868740 1 -- 172.21.15.83:0/2394147807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479160 msgr2=0x5612c3489810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.138+0000 7faa73868740 1 --2- 172.21.15.83:0/2394147807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479160 0x5612c3489810 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faa60009ab0 tx=0x7faa60009f70 comp rx=0 tx=0).stop 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.138+0000 7faa73868740 1 -- 172.21.15.83:0/2394147807 shutdown_connections 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.138+0000 7faa73868740 1 --2- 172.21.15.83:0/2394147807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479160 0x5612c3489810 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.138+0000 7faa73868740 1 --2- 172.21.15.83:0/2394147807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3479b20 0x5612c34822d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.138+0000 7faa73868740 1 --2- 172.21.15.83:0/2394147807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3477f90 0x5612c3480ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.138+0000 7faa73868740 1 -- 172.21.15.83:0/2394147807 >> 172.21.15.83:0/2394147807 conn(0x5612c346f070 msgr2=0x5612c346a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.138+0000 7faa73868740 1 -- 172.21.15.83:0/2394147807 shutdown_connections 2024-04-25T11:43:32.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.138+0000 7faa73868740 1 -- 172.21.15.83:0/2394147807 wait complete. 2024-04-25T11:43:32.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.138+0000 7faa73868740 1 -- start start 2024-04-25T11:43:32.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.138+0000 7faa73868740 1 Processor -- start 2024-04-25T11:43:32.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa73868740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3477f90 0x5612c347dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa73868740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3479160 0x5612c3489770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa73868740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c347e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa73868740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5612c34901d0 con 0x5612c3479b20 2024-04-25T11:43:32.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa73868740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5612c348fb00 con 0x5612c3477f90 2024-04-25T11:43:32.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa73868740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5612c34908a0 con 0x5612c3479160 2024-04-25T11:43:32.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa73868740 10 client.0 _ll_register_callbacks cb 0x7faa6404a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:32.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa6ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c347e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa6ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c347e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48824/0 (socket says 172.21.15.83:48824) 2024-04-25T11:43:32.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa6ad9d640 1 -- 172.21.15.83:0/3343922590 learned_addr learned my addr 172.21.15.83:0/3343922590 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa6a59c640 1 --2- 172.21.15.83:0/3343922590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3477f90 0x5612c347dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa69d9b640 1 --2- 172.21.15.83:0/3343922590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3479160 0x5612c3489770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa6ad9d640 1 -- 172.21.15.83:0/3343922590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3479160 msgr2=0x5612c3489770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa6ad9d640 1 --2- 172.21.15.83:0/3343922590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3479160 0x5612c3489770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa6ad9d640 1 -- 172.21.15.83:0/3343922590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3477f90 msgr2=0x5612c347dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa6ad9d640 1 --2- 172.21.15.83:0/3343922590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3477f90 0x5612c347dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa6ad9d640 1 -- 172.21.15.83:0/3343922590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612c347c7d0 con 0x5612c3479b20 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa69d9b640 1 --2- 172.21.15.83:0/3343922590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3479160 0x5612c3489770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa6ad9d640 1 --2- 172.21.15.83:0/3343922590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c347e1c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faa64031a20 tx=0x7faa6402fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613bded1be55490a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa51ffb640 1 -- 172.21.15.83:0/3343922590 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa64059070 con 0x5612c3479b20 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa73868740 1 -- 172.21.15.83:0/3343922590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa60014ca0 con 0x5612c3479b20 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa51ffb640 1 -- 172.21.15.83:0/3343922590 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7faa6402fcc0 con 0x5612c3479b20 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa51ffb640 1 -- 172.21.15.83:0/3343922590 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faa6406f050 con 0x5612c3479b20 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa73868740 10 client.15282 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:32.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.139+0000 7faa73868740 1 -- 172.21.15.83:0/3343922590 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5612c34f4e90 con 0x5612c3479b20 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.140+0000 7faa51ffb640 1 -- 172.21.15.83:0/3343922590 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa60014ca0 con 0x5612c3479b20 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.140+0000 7faa51ffb640 1 -- 172.21.15.83:0/3343922590 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7faa640531d0 con 0x5612c3479b20 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.140+0000 7faa73868740 10 client.15282 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.140+0000 7faa73868740 10 client.15282 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.140+0000 7faa73868740 20 client.15282 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.140+0000 7faa73868740 1 --2- 172.21.15.83:0/3343922590 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5612c34f9810 0x5612c34f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.140+0000 7faa73868740 4 client.15282 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.140+0000 7faa73868740 1 -- 172.21.15.83:0/3343922590 --> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] -- command(tid 0: {"prefix": "status"}) -- 0x5612c33e04e0 con 0x5612c34f9810 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.140+0000 7faa6a59c640 1 --2- 172.21.15.83:0/3343922590 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5612c34f9810 0x5612c34f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.140+0000 7faa51ffb640 10 client.15282 ms_handle_connect on v2:172.21.15.94:6836/2475976592 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.140+0000 7faa6a59c640 1 --2- 172.21.15.83:0/3343922590 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5612c34f9810 0x5612c34f5c50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=18c1c3ebcf44c144 server_cookie=f440a94b31565f01 in_seq=0 out_seq=0 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa51ffb640 1 -- 172.21.15.83:0/3343922590 <== mds.3 v2:172.21.15.94:6836/2475976592 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5612c33e04e0 con 0x5612c34f9810 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa51ffb640 10 client.15282 handle_command_reply: tid=0 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa73868740 1 client.15282 shutdown 2024-04-25T11:43:32.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa73868740 1 -- 172.21.15.83:0/3343922590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 msgr2=0x5612c347e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa73868740 1 --2- 172.21.15.83:0/3343922590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c347e1c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faa64031a20 tx=0x7faa6402fa00 comp rx=0 tx=0).stop 2024-04-25T11:43:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa73868740 1 -- 172.21.15.83:0/3343922590 shutdown_connections 2024-04-25T11:43:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa73868740 1 --2- 172.21.15.83:0/3343922590 >> [v2:172.21.15.94:6836/2475976592,v1:172.21.15.94:6837/2475976592] conn(0x5612c34f9810 0x5612c34f5c50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa73868740 1 --2- 172.21.15.83:0/3343922590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612c3479b20 0x5612c347e1c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa51ffb640 0 client.15282 ms_handle_reset on v2:172.21.15.94:6836/2475976592 2024-04-25T11:43:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa73868740 1 --2- 172.21.15.83:0/3343922590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5612c3479160 0x5612c3489770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa73868740 1 --2- 172.21.15.83:0/3343922590 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5612c3477f90 0x5612c347dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa73868740 1 -- 172.21.15.83:0/3343922590 >> 172.21.15.83:0/3343922590 conn(0x5612c346f070 msgr2=0x5612c34822d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa73868740 1 -- 172.21.15.83:0/3343922590 shutdown_connections 2024-04-25T11:43:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa73868740 1 -- 172.21.15.83:0/3343922590 wait complete. 2024-04-25T11:43:32.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa73868740 20 client.15282 trim_cache size 0 max 16384 2024-04-25T11:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "7fcf54c6-02f7-11ef-bc93-c7b262605968", 2024-04-25T11:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 3, 2024-04-25T11:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24440, 2024-04-25T11:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T11:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T11:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T11:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 102.532239838, 2024-04-25T11:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-04-25T11:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-25T11:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T11:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 123.158414167 2024-04-25T11:43:32.135 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:43:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa70b44640 1 -- 172.21.15.83:0/4230532117 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faa540a3bb0 msgr2=0x7faa540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa70b44640 1 --2- 172.21.15.83:0/4230532117 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faa540a3bb0 0x7faa540a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faa6404b000 tx=0x7faa64048000 comp rx=0 tx=0).stop 2024-04-25T11:43:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa70b44640 1 -- 172.21.15.83:0/4230532117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c100e80 msgr2=0x7faa6c198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa70b44640 1 --2- 172.21.15.83:0/4230532117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c100e80 0x7faa6c198230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00c970 tx=0x7faa5c00cd70 comp rx=0 tx=0).stop 2024-04-25T11:43:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa70b44640 1 -- 172.21.15.83:0/4230532117 shutdown_connections 2024-04-25T11:43:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa70b44640 1 --2- 172.21.15.83:0/4230532117 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7faa540a3bb0 0x7faa540a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa70b44640 1 --2- 172.21.15.83:0/4230532117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa6c102a20 0x7faa6c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa70b44640 1 --2- 172.21.15.83:0/4230532117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa6c102060 0x7faa6c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa70b44640 1 --2- 172.21.15.83:0/4230532117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa6c100e80 0x7faa6c198230 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa70b44640 1 -- 172.21.15.83:0/4230532117 >> 172.21.15.83:0/4230532117 conn(0x7faa6c0fc440 msgr2=0x7faa6c0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa70b44640 1 -- 172.21.15.83:0/4230532117 shutdown_connections 2024-04-25T11:43:32.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.141+0000 7faa70b44640 1 -- 172.21.15.83:0/4230532117 wait complete. 2024-04-25T11:43:32.138 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T11:43:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.250+0000 7f5e04730640 1 -- 172.21.15.83:0/4102806529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc101fe0 msgr2=0x7f5dfc102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.250+0000 7f5e04730640 1 --2- 172.21.15.83:0/4102806529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc101fe0 0x7f5dfc102460 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009900 tx=0x7f5df002a410 comp rx=0 tx=0).stop 2024-04-25T11:43:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e04730640 1 -- 172.21.15.83:0/4102806529 shutdown_connections 2024-04-25T11:43:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e04730640 1 --2- 172.21.15.83:0/4102806529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc101fe0 0x7f5dfc102460 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e04730640 1 --2- 172.21.15.83:0/4102806529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dfc1029a0 0x7f5dfc1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e04730640 1 --2- 172.21.15.83:0/4102806529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dfc100e90 0x7f5dfc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e04730640 1 -- 172.21.15.83:0/4102806529 >> 172.21.15.83:0/4102806529 conn(0x7f5dfc0fc400 msgr2=0x7f5dfc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e04730640 1 -- 172.21.15.83:0/4102806529 shutdown_connections 2024-04-25T11:43:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e04730640 1 -- 172.21.15.83:0/4102806529 wait complete. 2024-04-25T11:43:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e04730640 1 Processor -- start 2024-04-25T11:43:32.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e04730640 1 -- start start 2024-04-25T11:43:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e04730640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dfc100e90 0x7f5dfc069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e04730640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc101fe0 0x7f5dfc19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e04730640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dfc1029a0 0x7f5dfc19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e04730640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc10c9d0 con 0x7f5dfc101fe0 2024-04-25T11:43:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e04730640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc10c850 con 0x7f5dfc1029a0 2024-04-25T11:43:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e04730640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc10cb50 con 0x7f5dfc100e90 2024-04-25T11:43:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e01ccb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc101fe0 0x7f5dfc19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e01ccb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc101fe0 0x7f5dfc19cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48852/0 (socket says 172.21.15.83:48852) 2024-04-25T11:43:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e01ccb640 1 -- 172.21.15.83:0/4256559495 learned_addr learned my addr 172.21.15.83:0/4256559495 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.251+0000 7f5e024cc640 1 --2- 172.21.15.83:0/4256559495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dfc100e90 0x7f5dfc069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.252+0000 7f5e02ccd640 1 --2- 172.21.15.83:0/4256559495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dfc1029a0 0x7f5dfc19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.252+0000 7f5e01ccb640 1 -- 172.21.15.83:0/4256559495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dfc100e90 msgr2=0x7f5dfc069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.252+0000 7f5e01ccb640 1 --2- 172.21.15.83:0/4256559495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dfc100e90 0x7f5dfc069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.252+0000 7f5e01ccb640 1 -- 172.21.15.83:0/4256559495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dfc1029a0 msgr2=0x7f5dfc19f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.252+0000 7f5e01ccb640 1 --2- 172.21.15.83:0/4256559495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dfc1029a0 0x7f5dfc19f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.252+0000 7f5e01ccb640 1 -- 172.21.15.83:0/4256559495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dfc19fe30 con 0x7f5dfc101fe0 2024-04-25T11:43:32.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.252+0000 7f5e02ccd640 1 --2- 172.21.15.83:0/4256559495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dfc1029a0 0x7f5dfc19f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:32.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.252+0000 7f5e01ccb640 1 --2- 172.21.15.83:0/4256559495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc101fe0 0x7f5dfc19cf30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5df0006010 tx=0x7f5df00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.252+0000 7f5dfb7fe640 1 -- 172.21.15.83:0/4256559495 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df003f030 con 0x7f5dfc101fe0 2024-04-25T11:43:32.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.252+0000 7f5e04730640 1 -- 172.21.15.83:0/4256559495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dfc1a00c0 con 0x7f5dfc101fe0 2024-04-25T11:43:32.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.252+0000 7f5dfb7fe640 1 -- 172.21.15.83:0/4256559495 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5df0003cb0 con 0x7f5dfc101fe0 2024-04-25T11:43:32.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.252+0000 7f5dfb7fe640 1 -- 172.21.15.83:0/4256559495 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df0035070 con 0x7f5dfc101fe0 2024-04-25T11:43:32.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.252+0000 7f5e04730640 1 -- 172.21.15.83:0/4256559495 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dfc1a0600 con 0x7f5dfc101fe0 2024-04-25T11:43:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.254+0000 7f5dfb7fe640 1 -- 172.21.15.83:0/4256559495 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5df0042030 con 0x7f5dfc101fe0 2024-04-25T11:43:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.254+0000 7f5e07447740 1 Processor -- start 2024-04-25T11:43:32.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e07447740 1 -- start start 2024-04-25T11:43:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e07447740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc947b80 0x5569cc947fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e07447740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc948d50 0x5569cc9491d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e07447740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc949710 0x5569cc94ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e07447740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5569cc8acc50 con 0x5569cc948d50 2024-04-25T11:43:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e07447740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5569cc849830 con 0x5569cc949710 2024-04-25T11:43:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e07447740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5569cc8a8c10 con 0x5569cc947b80 2024-04-25T11:43:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5dfb7fe640 1 --2- 172.21.15.83:0/4256559495 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5de40a3bb0 0x7f5de40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e024cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc948d50 0x5569cc9491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e024cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc948d50 0x5569cc9491d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48864/0 (socket says 172.21.15.83:48864) 2024-04-25T11:43:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e024cc640 1 -- 172.21.15.83:0/3638851017 learned_addr learned my addr 172.21.15.83:0/3638851017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e02ccd640 1 --2- 172.21.15.83:0/4256559495 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5de40a3bb0 0x7f5de40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e01ccb640 1 --2- 172.21.15.83:0/3638851017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc949710 0x5569cc94ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e02ccd640 1 --2- 172.21.15.83:0/3638851017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc947b80 0x5569cc947fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5dfb7fe640 1 -- 172.21.15.83:0/4256559495 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5df00f0a30 con 0x7f5dfc101fe0 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e024cc640 1 -- 172.21.15.83:0/3638851017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc947b80 msgr2=0x5569cc947fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e024cc640 1 --2- 172.21.15.83:0/3638851017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc947b80 0x5569cc947fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e024cc640 1 -- 172.21.15.83:0/3638851017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc949710 msgr2=0x5569cc94ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e024cc640 1 --2- 172.21.15.83:0/3638851017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc949710 0x5569cc94ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e024cc640 1 -- 172.21.15.83:0/3638851017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569cc5bf2d0 con 0x5569cc948d50 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.255+0000 7f5e02ccd640 1 --2- 172.21.15.83:0/3638851017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc947b80 0x5569cc947fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.256+0000 7f5e02ccd640 1 --2- 172.21.15.83:0/4256559495 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5de40a3bb0 0x7f5de40a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5569cc948c50 tx=0x7f5dec005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.256+0000 7f5e024cc640 1 --2- 172.21.15.83:0/3638851017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc948d50 0x5569cc9491d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5df40039f0 tx=0x7f5df4003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3f9d33e1d8700e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.256+0000 7f5df8ff9640 1 -- 172.21.15.83:0/3638851017 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4004b20 con 0x5569cc948d50 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.256+0000 7f5df8ff9640 1 -- 172.21.15.83:0/3638851017 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5df4004cc0 con 0x5569cc948d50 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.256+0000 7f5df8ff9640 1 -- 172.21.15.83:0/3638851017 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40126b0 con 0x5569cc948d50 2024-04-25T11:43:32.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.256+0000 7f5e07447740 1 -- 172.21.15.83:0/3638851017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc948d50 msgr2=0x5569cc9491d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.256+0000 7f5e07447740 1 --2- 172.21.15.83:0/3638851017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc948d50 0x5569cc9491d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5df40039f0 tx=0x7f5df4003eb0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.256+0000 7f5e07447740 1 -- 172.21.15.83:0/3638851017 shutdown_connections 2024-04-25T11:43:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.256+0000 7f5e07447740 1 --2- 172.21.15.83:0/3638851017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc948d50 0x5569cc9491d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.256+0000 7f5e07447740 1 --2- 172.21.15.83:0/3638851017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc949710 0x5569cc94ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.256+0000 7f5e07447740 1 --2- 172.21.15.83:0/3638851017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc947b80 0x5569cc947fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.256+0000 7f5e07447740 1 -- 172.21.15.83:0/3638851017 >> 172.21.15.83:0/3638851017 conn(0x5569cc943070 msgr2=0x5569cc943470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e07447740 1 -- 172.21.15.83:0/3638851017 shutdown_connections 2024-04-25T11:43:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e07447740 1 -- 172.21.15.83:0/3638851017 wait complete. 2024-04-25T11:43:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e07447740 1 -- start start 2024-04-25T11:43:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e07447740 1 Processor -- start 2024-04-25T11:43:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e07447740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc947b80 0x5569cc951f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e07447740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc948d50 0x5569cc9524c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e07447740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc949710 0x5569cc959500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e07447740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5569cc950720 con 0x5569cc949710 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e07447740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5569cc847470 con 0x5569cc947b80 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e07447740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5569cc9508a0 con 0x5569cc948d50 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e02ccd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc949710 0x5569cc959500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e07447740 10 client.0 _ll_register_callbacks cb 0x5569cc86cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e02ccd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc949710 0x5569cc959500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48880/0 (socket says 172.21.15.83:48880) 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e02ccd640 1 -- 172.21.15.83:0/4192278396 learned_addr learned my addr 172.21.15.83:0/4192278396 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e024cc640 1 --2- 172.21.15.83:0/4192278396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc947b80 0x5569cc951f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e02ccd640 1 -- 172.21.15.83:0/4192278396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc948d50 msgr2=0x5569cc9524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e02ccd640 1 --2- 172.21.15.83:0/4192278396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc948d50 0x5569cc9524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e02ccd640 1 -- 172.21.15.83:0/4192278396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc947b80 msgr2=0x5569cc951f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e02ccd640 1 --2- 172.21.15.83:0/4192278396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc947b80 0x5569cc951f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.257+0000 7f5e02ccd640 1 -- 172.21.15.83:0/4192278396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569cc6896b0 con 0x5569cc949710 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5e02ccd640 1 --2- 172.21.15.83:0/4192278396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc949710 0x5569cc959500 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0117c0 tx=0x7f5dec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a867c765c90ea34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5de1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:32.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5de1ffb640 1 -- 172.21.15.83:0/4192278396 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec010b60 con 0x5569cc949710 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5de1ffb640 1 -- 172.21.15.83:0/4192278396 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5dec010d00 con 0x5569cc949710 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5e07447740 1 -- 172.21.15.83:0/4192278396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5df4004cc0 con 0x5569cc949710 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5de1ffb640 1 -- 172.21.15.83:0/4192278396 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5dec026400 con 0x5569cc949710 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5e07447740 10 client.15294 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5e07447740 1 -- 172.21.15.83:0/4192278396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5569cc95c940 con 0x5569cc949710 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5de1ffb640 1 -- 172.21.15.83:0/4192278396 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5df4004cc0 con 0x5569cc949710 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5de1ffb640 1 -- 172.21.15.83:0/4192278396 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f5dec0315b0 con 0x5569cc949710 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5e07447740 10 client.15294 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5e07447740 10 client.15294 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5e07447740 20 client.15294 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5e07447740 1 --2- 172.21.15.83:0/4192278396 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5569cc95f050 0x5569cc9604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5e07447740 4 client.15294 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.258+0000 7f5e07447740 1 -- 172.21.15.83:0/4192278396 --> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569cc8ba670 con 0x5569cc95f050 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.259+0000 7f5e024cc640 1 --2- 172.21.15.83:0/4192278396 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5569cc95f050 0x5569cc9604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.259+0000 7f5e024cc640 1 --2- 172.21.15.83:0/4192278396 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5569cc95f050 0x5569cc9604c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6f800fecc36962d server_cookie=f8336cfc0fa64592 in_seq=0 out_seq=0 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.259+0000 7f5de1ffb640 10 client.15294 ms_handle_connect on v2:172.21.15.111:6836/2927430878 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.260+0000 7f5de1ffb640 1 -- 172.21.15.83:0/4192278396 <== mds.2 v2:172.21.15.111:6836/2927430878 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5569cc8ba670 con 0x5569cc95f050 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.260+0000 7f5de1ffb640 10 client.15294 handle_command_reply: tid=0 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.260+0000 7f5e07447740 1 client.15294 shutdown 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.260+0000 7f5e07447740 1 -- 172.21.15.83:0/4192278396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc949710 msgr2=0x5569cc959500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.260+0000 7f5e07447740 1 --2- 172.21.15.83:0/4192278396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc949710 0x5569cc959500 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0117c0 tx=0x7f5dec0046c0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.260+0000 7f5e07447740 1 -- 172.21.15.83:0/4192278396 shutdown_connections 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.260+0000 7f5e07447740 1 --2- 172.21.15.83:0/4192278396 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5569cc95f050 0x5569cc9604c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.260+0000 7f5e07447740 1 --2- 172.21.15.83:0/4192278396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc949710 0x5569cc959500 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.260+0000 7f5e07447740 1 --2- 172.21.15.83:0/4192278396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc948d50 0x5569cc9524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.260+0000 7f5de1ffb640 0 client.15294 ms_handle_reset on v2:172.21.15.111:6836/2927430878 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.260+0000 7f5e07447740 1 --2- 172.21.15.83:0/4192278396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc947b80 0x5569cc951f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.260+0000 7f5e07447740 1 -- 172.21.15.83:0/4192278396 >> 172.21.15.83:0/4192278396 conn(0x5569cc943070 msgr2=0x5569cc9421d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.260+0000 7f5e07447740 1 -- 172.21.15.83:0/4192278396 shutdown_connections 2024-04-25T11:43:32.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.260+0000 7f5e07447740 1 -- 172.21.15.83:0/4192278396 wait complete. 2024-04-25T11:43:32.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.260+0000 7f5e07447740 20 client.15294 trim_cache size 0 max 16384 2024-04-25T11:43:32.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e07447740 1 Processor -- start 2024-04-25T11:43:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e07447740 1 -- start start 2024-04-25T11:43:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e07447740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc947b80 0x5569cc94c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e07447740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc948d50 0x5569cc9594c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e07447740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc949710 0x5569cc94ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e07447740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5569cc95e800 con 0x5569cc947b80 2024-04-25T11:43:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e07447740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5569cc95eed0 con 0x5569cc949710 2024-04-25T11:43:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e07447740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5569cc95f5a0 con 0x5569cc948d50 2024-04-25T11:43:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e024cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc947b80 0x5569cc94c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e024cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc947b80 0x5569cc94c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48894/0 (socket says 172.21.15.83:48894) 2024-04-25T11:43:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e024cc640 1 -- 172.21.15.83:0/1352155467 learned_addr learned my addr 172.21.15.83:0/1352155467 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e01ccb640 1 --2- 172.21.15.83:0/1352155467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc948d50 0x5569cc9594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e024cc640 1 -- 172.21.15.83:0/1352155467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc948d50 msgr2=0x5569cc9594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e024cc640 1 --2- 172.21.15.83:0/1352155467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc948d50 0x5569cc9594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e024cc640 1 -- 172.21.15.83:0/1352155467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc949710 msgr2=0x5569cc94ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e024cc640 1 --2- 172.21.15.83:0/1352155467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc949710 0x5569cc94ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e024cc640 1 -- 172.21.15.83:0/1352155467 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569cc9487d0 con 0x5569cc947b80 2024-04-25T11:43:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e01ccb640 1 --2- 172.21.15.83:0/1352155467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc948d50 0x5569cc9594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5e024cc640 1 --2- 172.21.15.83:0/1352155467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc947b80 0x5569cc94c790 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x5569cc943f40 tx=0x7f5df4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccd1a78c7d7fe5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.278+0000 7f5de3fff640 1 -- 172.21.15.83:0/1352155467 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df402f070 con 0x5569cc947b80 2024-04-25T11:43:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.279+0000 7f5de3fff640 1 -- 172.21.15.83:0/1352155467 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5df4004410 con 0x5569cc947b80 2024-04-25T11:43:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.279+0000 7f5de3fff640 1 -- 172.21.15.83:0/1352155467 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40127a0 con 0x5569cc947b80 2024-04-25T11:43:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.279+0000 7f5e07447740 1 -- 172.21.15.83:0/1352155467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc947b80 msgr2=0x5569cc94c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.279+0000 7f5e07447740 1 --2- 172.21.15.83:0/1352155467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc947b80 0x5569cc94c790 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x5569cc943f40 tx=0x7f5df4005fd0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.279+0000 7f5e07447740 1 -- 172.21.15.83:0/1352155467 shutdown_connections 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.279+0000 7f5e07447740 1 --2- 172.21.15.83:0/1352155467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc949710 0x5569cc94ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.279+0000 7f5e07447740 1 --2- 172.21.15.83:0/1352155467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc948d50 0x5569cc9594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.279+0000 7f5e07447740 1 --2- 172.21.15.83:0/1352155467 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc947b80 0x5569cc94c790 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.279+0000 7f5e07447740 1 -- 172.21.15.83:0/1352155467 >> 172.21.15.83:0/1352155467 conn(0x5569cc943070 msgr2=0x5569cc93e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.279+0000 7f5e07447740 1 -- 172.21.15.83:0/1352155467 shutdown_connections 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.279+0000 7f5e07447740 1 -- 172.21.15.83:0/1352155467 wait complete. 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.279+0000 7f5e07447740 1 -- start start 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.279+0000 7f5e07447740 1 Processor -- start 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e07447740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc947b80 0x5569cc954f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e07447740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc948d50 0x5569cc955450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e07447740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc949710 0x5569cc958470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e07447740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5569cc960340 con 0x5569cc947b80 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e07447740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x5569cc95fc70 con 0x5569cc949710 2024-04-25T11:43:32.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e07447740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5569cc960a10 con 0x5569cc948d50 2024-04-25T11:43:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e07447740 10 client.0 _ll_register_callbacks cb 0x7f5dec00df90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e024cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc947b80 0x5569cc954f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e024cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc947b80 0x5569cc954f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48898/0 (socket says 172.21.15.83:48898) 2024-04-25T11:43:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e024cc640 1 -- 172.21.15.83:0/23779868 learned_addr learned my addr 172.21.15.83:0/23779868 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e01ccb640 1 --2- 172.21.15.83:0/23779868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc948d50 0x5569cc955450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e024cc640 1 -- 172.21.15.83:0/23779868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc948d50 msgr2=0x5569cc955450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e024cc640 1 --2- 172.21.15.83:0/23779868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc948d50 0x5569cc955450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e024cc640 1 -- 172.21.15.83:0/23779868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc949710 msgr2=0x5569cc958470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e024cc640 1 --2- 172.21.15.83:0/23779868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc949710 0x5569cc958470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e024cc640 1 -- 172.21.15.83:0/23779868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569cc958a60 con 0x5569cc947b80 2024-04-25T11:43:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e024cc640 1 --2- 172.21.15.83:0/23779868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc947b80 0x5569cc954f10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5df4004930 tx=0x7f5df4004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383f8900c579da49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5de1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5de1ffb640 1 -- 172.21.15.83:0/23779868 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4038060 con 0x5569cc947b80 2024-04-25T11:43:32.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5de1ffb640 1 -- 172.21.15.83:0/23779868 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f5df402a2f0 con 0x5569cc947b80 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e07447740 1 -- 172.21.15.83:0/23779868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5df4004410 con 0x5569cc947b80 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5de1ffb640 1 -- 172.21.15.83:0/23779868 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5df402f040 con 0x5569cc947b80 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e07447740 10 client.15300 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.280+0000 7f5e07447740 1 -- 172.21.15.83:0/23779868 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5569cc9c52d0 con 0x5569cc947b80 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.281+0000 7f5de1ffb640 1 -- 172.21.15.83:0/23779868 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5df4004410 con 0x5569cc947b80 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.281+0000 7f5de1ffb640 1 -- 172.21.15.83:0/23779868 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f5df4032070 con 0x5569cc947b80 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.281+0000 7f5e07447740 10 client.15300 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.281+0000 7f5e07447740 10 client.15300 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.281+0000 7f5e07447740 20 client.15300 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.281+0000 7f5e07447740 1 --2- 172.21.15.83:0/23779868 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5569cc9c9800 0x5569cc9c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.281+0000 7f5e07447740 4 client.15300 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.281+0000 7f5e07447740 1 -- 172.21.15.83:0/23779868 --> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] -- command(tid 0: {"prefix": "status"}) -- 0x5569cc8b44e0 con 0x5569cc9c9800 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.281+0000 7f5e01ccb640 1 --2- 172.21.15.83:0/23779868 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5569cc9c9800 0x5569cc9c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5de1ffb640 10 client.15300 ms_handle_connect on v2:172.21.15.111:6836/2927430878 2024-04-25T11:43:32.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e01ccb640 1 --2- 172.21.15.83:0/23779868 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5569cc9c9800 0x5569cc9c8510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be47f346d24c77e5 server_cookie=e871a2b0d71d1915 in_seq=0 out_seq=0 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5de1ffb640 1 -- 172.21.15.83:0/23779868 <== mds.2 v2:172.21.15.111:6836/2927430878 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5569cc8b44e0 con 0x5569cc9c9800 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5de1ffb640 10 client.15300 handle_command_reply: tid=0 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e07447740 1 client.15300 shutdown 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e07447740 1 -- 172.21.15.83:0/23779868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc947b80 msgr2=0x5569cc954f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e07447740 1 --2- 172.21.15.83:0/23779868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc947b80 0x5569cc954f10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5df4004930 tx=0x7f5df4004960 comp rx=0 tx=0).stop 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e07447740 1 -- 172.21.15.83:0/23779868 shutdown_connections 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e07447740 1 --2- 172.21.15.83:0/23779868 >> [v2:172.21.15.111:6836/2927430878,v1:172.21.15.111:6837/2927430878] conn(0x5569cc9c9800 0x5569cc9c8510 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e07447740 1 --2- 172.21.15.83:0/23779868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5569cc949710 0x5569cc958470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5de1ffb640 0 client.15300 ms_handle_reset on v2:172.21.15.111:6836/2927430878 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e07447740 1 --2- 172.21.15.83:0/23779868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5569cc948d50 0x5569cc955450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e07447740 1 --2- 172.21.15.83:0/23779868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569cc947b80 0x5569cc954f10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e07447740 1 -- 172.21.15.83:0/23779868 >> 172.21.15.83:0/23779868 conn(0x5569cc943070 msgr2=0x5569cc94c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e07447740 1 -- 172.21.15.83:0/23779868 shutdown_connections 2024-04-25T11:43:32.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e07447740 1 -- 172.21.15.83:0/23779868 wait complete. 2024-04-25T11:43:32.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e07447740 20 client.15300 trim_cache size 0 max 16384 2024-04-25T11:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "7fcf54c6-02f7-11ef-bc93-c7b262605968", 2024-04-25T11:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 2, 2024-04-25T11:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24442, 2024-04-25T11:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T11:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T11:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T11:43:32.276 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 103.678689843, 2024-04-25T11:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-04-25T11:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-25T11:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T11:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 126.112313911 2024-04-25T11:43:32.277 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:43:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e04730640 1 -- 172.21.15.83:0/4256559495 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5de40a3bb0 msgr2=0x7f5de40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e04730640 1 --2- 172.21.15.83:0/4256559495 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5de40a3bb0 0x7f5de40a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x5569cc948c50 tx=0x7f5dec005da0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e04730640 1 -- 172.21.15.83:0/4256559495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc101fe0 msgr2=0x7f5dfc19cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.282+0000 7f5e04730640 1 --2- 172.21.15.83:0/4256559495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc101fe0 0x7f5dfc19cf30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5df0006010 tx=0x7f5df00042f0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.283+0000 7f5e04730640 1 -- 172.21.15.83:0/4256559495 shutdown_connections 2024-04-25T11:43:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.283+0000 7f5e04730640 1 --2- 172.21.15.83:0/4256559495 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f5de40a3bb0 0x7f5de40a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.283+0000 7f5e04730640 1 --2- 172.21.15.83:0/4256559495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc101fe0 0x7f5dfc19cf30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.283+0000 7f5e04730640 1 --2- 172.21.15.83:0/4256559495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dfc1029a0 0x7f5dfc19f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.283+0000 7f5e04730640 1 --2- 172.21.15.83:0/4256559495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5dfc100e90 0x7f5dfc069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.283+0000 7f5e04730640 1 -- 172.21.15.83:0/4256559495 >> 172.21.15.83:0/4256559495 conn(0x7f5dfc0fc400 msgr2=0x7f5dfc105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.283+0000 7f5e04730640 1 -- 172.21.15.83:0/4256559495 shutdown_connections 2024-04-25T11:43:32.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.283+0000 7f5e04730640 1 -- 172.21.15.83:0/4256559495 wait complete. 2024-04-25T11:43:32.279 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T11:43:32.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:32 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/835315641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:43:32.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:32 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/915005318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:43:32.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:32 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2148792051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:32.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.392+0000 7fd31d74f640 1 -- 172.21.15.83:0/4254311732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318100f30 msgr2=0x7fd318101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.392+0000 7fd31d74f640 1 --2- 172.21.15.83:0/4254311732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318100f30 0x7fd318101350 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd308009900 tx=0x7fd30802a410 comp rx=0 tx=0).stop 2024-04-25T11:43:32.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.392+0000 7fd31d74f640 1 -- 172.21.15.83:0/4254311732 shutdown_connections 2024-04-25T11:43:32.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.392+0000 7fd31d74f640 1 --2- 172.21.15.83:0/4254311732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd318102a40 0x7fd318109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.392+0000 7fd31d74f640 1 --2- 172.21.15.83:0/4254311732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd318102080 0x7fd318102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.392+0000 7fd31d74f640 1 --2- 172.21.15.83:0/4254311732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318100f30 0x7fd318101350 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.392+0000 7fd31d74f640 1 -- 172.21.15.83:0/4254311732 >> 172.21.15.83:0/4254311732 conn(0x7fd3180fc4a0 msgr2=0x7fd3180fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.392+0000 7fd31d74f640 1 -- 172.21.15.83:0/4254311732 shutdown_connections 2024-04-25T11:43:32.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.392+0000 7fd31d74f640 1 -- 172.21.15.83:0/4254311732 wait complete. 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.392+0000 7fd31d74f640 1 Processor -- start 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd31d74f640 1 -- start start 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd31d74f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318100f30 0x7fd3181971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd31d74f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd318102080 0x7fd318197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd31d74f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd318102a40 0x7fd31819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd31d74f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd31810cb80 con 0x7fd318100f30 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd31d74f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd31810ca00 con 0x7fd318102080 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd31d74f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd31810cd00 con 0x7fd318102a40 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd316ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318100f30 0x7fd3181971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd316ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318100f30 0x7fd3181971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48908/0 (socket says 172.21.15.83:48908) 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd316ffd640 1 -- 172.21.15.83:0/1426597678 learned_addr learned my addr 172.21.15.83:0/1426597678 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd3177fe640 1 --2- 172.21.15.83:0/1426597678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd318102a40 0x7fd31819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd316ffd640 1 -- 172.21.15.83:0/1426597678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd318102a40 msgr2=0x7fd31819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd316ffd640 1 --2- 172.21.15.83:0/1426597678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd318102a40 0x7fd31819e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd316ffd640 1 -- 172.21.15.83:0/1426597678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd318102080 msgr2=0x7fd318197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd316ffd640 1 --2- 172.21.15.83:0/1426597678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd318102080 0x7fd318197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd316ffd640 1 -- 172.21.15.83:0/1426597678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31819ee10 con 0x7fd318100f30 2024-04-25T11:43:32.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd316ffd640 1 --2- 172.21.15.83:0/1426597678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318100f30 0x7fd3181971f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd308031500 tx=0x7fd3080043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd2fffff640 1 -- 172.21.15.83:0/1426597678 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30803f030 con 0x7fd318100f30 2024-04-25T11:43:32.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd31d74f640 1 -- 172.21.15.83:0/1426597678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31819f0a0 con 0x7fd318100f30 2024-04-25T11:43:32.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd2fffff640 1 -- 172.21.15.83:0/1426597678 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd308003800 con 0x7fd318100f30 2024-04-25T11:43:32.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd2fffff640 1 -- 172.21.15.83:0/1426597678 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd308035070 con 0x7fd318100f30 2024-04-25T11:43:32.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.393+0000 7fd31d74f640 1 -- 172.21.15.83:0/1426597678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31819f610 con 0x7fd318100f30 2024-04-25T11:43:32.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.396+0000 7fd32046f740 1 Processor -- start 2024-04-25T11:43:32.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.396+0000 7fd2fffff640 1 -- 172.21.15.83:0/1426597678 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd3080039b0 con 0x7fd318100f30 2024-04-25T11:43:32.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.396+0000 7fd32046f740 1 -- start start 2024-04-25T11:43:32.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.396+0000 7fd32046f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b9741b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.396+0000 7fd32046f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b974f60 0x55602b9753e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.396+0000 7fd32046f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b975920 0x55602b97c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.396+0000 7fd32046f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55602b8d6c50 con 0x55602b973d90 2024-04-25T11:43:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.396+0000 7fd32046f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55602b873830 con 0x55602b974f60 2024-04-25T11:43:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.396+0000 7fd32046f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55602b8d2c10 con 0x55602b975920 2024-04-25T11:43:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.396+0000 7fd3177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b9741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.396+0000 7fd3177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b9741b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48922/0 (socket says 172.21.15.83:48922) 2024-04-25T11:43:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.396+0000 7fd3177fe640 1 -- 172.21.15.83:0/592886790 learned_addr learned my addr 172.21.15.83:0/592886790 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.397+0000 7fd316ffd640 1 --2- 172.21.15.83:0/592886790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b974f60 0x55602b9753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.397+0000 7fd3177fe640 1 -- 172.21.15.83:0/592886790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b975920 msgr2=0x55602b97c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.397+0000 7fd3167fc640 1 --2- 172.21.15.83:0/592886790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b975920 0x55602b97c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.397+0000 7fd3177fe640 1 --2- 172.21.15.83:0/592886790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b975920 0x55602b97c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.397+0000 7fd3177fe640 1 -- 172.21.15.83:0/592886790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b974f60 msgr2=0x55602b9753e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.397+0000 7fd3177fe640 1 --2- 172.21.15.83:0/592886790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b974f60 0x55602b9753e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.397+0000 7fd2fffff640 1 --2- 172.21.15.83:0/1426597678 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd3000a3bb0 0x7fd3000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.397+0000 7fd3177fe640 1 -- 172.21.15.83:0/592886790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55602b5e92d0 con 0x55602b973d90 2024-04-25T11:43:32.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.397+0000 7fd3167fc640 1 --2- 172.21.15.83:0/592886790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b975920 0x55602b97c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.397+0000 7fd316ffd640 1 --2- 172.21.15.83:0/1426597678 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd3000a3bb0 0x7fd3000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.397+0000 7fd3177fe640 1 --2- 172.21.15.83:0/592886790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b9741b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd310009f20 tx=0x7fd31000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7234ed1e1bf7c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.397+0000 7fd2fffff640 1 -- 172.21.15.83:0/1426597678 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd3080f0ee0 con 0x7fd318100f30 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.397+0000 7fd2fd7fa640 1 -- 172.21.15.83:0/592886790 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31001a070 con 0x55602b973d90 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.397+0000 7fd2fd7fa640 1 -- 172.21.15.83:0/592886790 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3100158c0 con 0x55602b973d90 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.397+0000 7fd2fd7fa640 1 -- 172.21.15.83:0/592886790 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31000a820 con 0x55602b973d90 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 -- 172.21.15.83:0/592886790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 msgr2=0x55602b9741b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 --2- 172.21.15.83:0/592886790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b9741b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd310009f20 tx=0x7fd31000c8b0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd316ffd640 1 --2- 172.21.15.83:0/1426597678 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd3000a3bb0 0x7fd3000a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd3080b9630 tx=0x7fd3080f6040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 -- 172.21.15.83:0/592886790 shutdown_connections 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 --2- 172.21.15.83:0/592886790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b975920 0x55602b97c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 --2- 172.21.15.83:0/592886790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b974f60 0x55602b9753e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 --2- 172.21.15.83:0/592886790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b9741b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 -- 172.21.15.83:0/592886790 >> 172.21.15.83:0/592886790 conn(0x55602b96d070 msgr2=0x55602b96d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 -- 172.21.15.83:0/592886790 shutdown_connections 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 -- 172.21.15.83:0/592886790 wait complete. 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 -- start start 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 Processor -- start 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b97e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b974f60 0x55602b97e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b975920 0x55602b9857b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55602b97c930 con 0x55602b973d90 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55602b871470 con 0x55602b975920 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55602b97cab0 con 0x55602b974f60 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd32046f740 10 client.0 _ll_register_callbacks cb 0x55602b896f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.398+0000 7fd3177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b97e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd3177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b97e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48924/0 (socket says 172.21.15.83:48924) 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd3177fe640 1 -- 172.21.15.83:0/3016472564 learned_addr learned my addr 172.21.15.83:0/3016472564 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd3177fe640 1 -- 172.21.15.83:0/3016472564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b974f60 msgr2=0x55602b97e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd3167fc640 1 --2- 172.21.15.83:0/3016472564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b974f60 0x55602b97e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd3177fe640 1 --2- 172.21.15.83:0/3016472564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b974f60 0x55602b97e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd3177fe640 1 -- 172.21.15.83:0/3016472564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b975920 msgr2=0x55602b9857b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd3177fe640 1 --2- 172.21.15.83:0/3016472564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b975920 0x55602b9857b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd3177fe640 1 -- 172.21.15.83:0/3016472564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55602b6b36b0 con 0x55602b973d90 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd3167fc640 1 --2- 172.21.15.83:0/3016472564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b974f60 0x55602b97e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd3177fe640 1 --2- 172.21.15.83:0/3016472564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b97e2a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd310009a90 tx=0x7fd310015ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd2e1f7be9c36a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd2faff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd2faff5640 1 -- 172.21.15.83:0/3016472564 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31001a040 con 0x55602b973d90 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd2faff5640 1 -- 172.21.15.83:0/3016472564 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd31000a5a0 con 0x55602b973d90 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd32046f740 1 -- 172.21.15.83:0/3016472564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3100158c0 con 0x55602b973d90 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd2faff5640 1 -- 172.21.15.83:0/3016472564 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd310009b90 con 0x55602b973d90 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd32046f740 10 client.15312 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd32046f740 1 -- 172.21.15.83:0/3016472564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55602b988a90 con 0x55602b973d90 2024-04-25T11:43:32.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd2faff5640 1 -- 172.21.15.83:0/3016472564 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3100158c0 con 0x55602b973d90 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd2faff5640 1 -- 172.21.15.83:0/3016472564 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fd310020780 con 0x55602b973d90 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.399+0000 7fd32046f740 10 client.15312 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.400+0000 7fd32046f740 10 client.15312 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.400+0000 7fd32046f740 20 client.15312 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.400+0000 7fd32046f740 1 --2- 172.21.15.83:0/3016472564 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55602b98ad80 0x55602b98c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.400+0000 7fd32046f740 4 client.15312 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.400+0000 7fd32046f740 1 -- 172.21.15.83:0/3016472564 --> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55602b8e4670 con 0x55602b98ad80 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.400+0000 7fd3167fc640 1 --2- 172.21.15.83:0/3016472564 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55602b98ad80 0x55602b98c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.400+0000 7fd2faff5640 10 client.15312 ms_handle_connect on v2:172.21.15.111:6838/2233531151 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.400+0000 7fd3167fc640 1 --2- 172.21.15.83:0/3016472564 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55602b98ad80 0x55602b98c1f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aaf898f02221c1f0 server_cookie=a97af2666797402a in_seq=0 out_seq=0 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.401+0000 7fd2faff5640 1 -- 172.21.15.83:0/3016472564 <== mds.1 v2:172.21.15.111:6838/2233531151 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55602b8e4670 con 0x55602b98ad80 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.401+0000 7fd2faff5640 10 client.15312 handle_command_reply: tid=0 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.401+0000 7fd32046f740 1 client.15312 shutdown 2024-04-25T11:43:32.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.401+0000 7fd32046f740 1 -- 172.21.15.83:0/3016472564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 msgr2=0x55602b97e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.401+0000 7fd32046f740 1 --2- 172.21.15.83:0/3016472564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b97e2a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd310009a90 tx=0x7fd310015ca0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.401+0000 7fd32046f740 1 -- 172.21.15.83:0/3016472564 shutdown_connections 2024-04-25T11:43:32.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.401+0000 7fd32046f740 1 --2- 172.21.15.83:0/3016472564 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55602b98ad80 0x55602b98c1f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.401+0000 7fd32046f740 1 --2- 172.21.15.83:0/3016472564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b975920 0x55602b9857b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.401+0000 7fd32046f740 1 --2- 172.21.15.83:0/3016472564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b974f60 0x55602b97e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.401+0000 7fd2faff5640 0 client.15312 ms_handle_reset on v2:172.21.15.111:6838/2233531151 2024-04-25T11:43:32.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.401+0000 7fd32046f740 1 --2- 172.21.15.83:0/3016472564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b97e2a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.401+0000 7fd32046f740 1 -- 172.21.15.83:0/3016472564 >> 172.21.15.83:0/3016472564 conn(0x55602b96d070 msgr2=0x55602b96c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.401+0000 7fd32046f740 1 -- 172.21.15.83:0/3016472564 shutdown_connections 2024-04-25T11:43:32.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.401+0000 7fd32046f740 1 -- 172.21.15.83:0/3016472564 wait complete. 2024-04-25T11:43:32.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.401+0000 7fd32046f740 20 client.15312 trim_cache size 0 max 16384 2024-04-25T11:43:32.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.419+0000 7fd32046f740 1 Processor -- start 2024-04-25T11:43:32.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.419+0000 7fd32046f740 1 -- start start 2024-04-25T11:43:32.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd32046f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b973d90 0x55602b977f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd32046f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b974f60 0x55602b97deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd32046f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b975920 0x55602b97e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd32046f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55602b98a530 con 0x55602b974f60 2024-04-25T11:43:32.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd32046f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55602b98ac00 con 0x55602b975920 2024-04-25T11:43:32.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd32046f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55602b98b2d0 con 0x55602b973d90 2024-04-25T11:43:32.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd3167fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b974f60 0x55602b97deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd3167fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b974f60 0x55602b97deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48938/0 (socket says 172.21.15.83:48938) 2024-04-25T11:43:32.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd3167fc640 1 -- 172.21.15.83:0/123791204 learned_addr learned my addr 172.21.15.83:0/123791204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd3167fc640 1 -- 172.21.15.83:0/123791204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b973d90 msgr2=0x55602b977f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd3167fc640 1 --2- 172.21.15.83:0/123791204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b973d90 0x55602b977f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd3167fc640 1 -- 172.21.15.83:0/123791204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b975920 msgr2=0x55602b97e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd3167fc640 1 --2- 172.21.15.83:0/123791204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b975920 0x55602b97e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd3167fc640 1 -- 172.21.15.83:0/123791204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55602b9749e0 con 0x55602b974f60 2024-04-25T11:43:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd3167fc640 1 --2- 172.21.15.83:0/123791204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b974f60 0x55602b97deb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55602b96df40 tx=0x7fd30c0147e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5d4af284542f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd2fcff9640 1 -- 172.21.15.83:0/123791204 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c017350 con 0x55602b974f60 2024-04-25T11:43:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd2fcff9640 1 -- 172.21.15.83:0/123791204 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd30c01b030 con 0x55602b974f60 2024-04-25T11:43:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.420+0000 7fd2fcff9640 1 -- 172.21.15.83:0/123791204 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c012be0 con 0x55602b974f60 2024-04-25T11:43:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 -- 172.21.15.83:0/123791204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b974f60 msgr2=0x55602b97deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 --2- 172.21.15.83:0/123791204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b974f60 0x55602b97deb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55602b96df40 tx=0x7fd30c0147e0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 -- 172.21.15.83:0/123791204 shutdown_connections 2024-04-25T11:43:32.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 --2- 172.21.15.83:0/123791204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b974f60 0x55602b97deb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 --2- 172.21.15.83:0/123791204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b975920 0x55602b97e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 --2- 172.21.15.83:0/123791204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b973d90 0x55602b977f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 -- 172.21.15.83:0/123791204 >> 172.21.15.83:0/123791204 conn(0x55602b96d070 msgr2=0x55602b9680b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 -- 172.21.15.83:0/123791204 shutdown_connections 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 -- 172.21.15.83:0/123791204 wait complete. 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 -- start start 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 Processor -- start 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b97a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b974f60 0x55602b97a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b975920 0x55602b9836c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55602b98c070 con 0x55602b973d90 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55602b98b9a0 con 0x55602b974f60 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55602b98c740 con 0x55602b975920 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd32046f740 10 client.0 _ll_register_callbacks cb 0x7fd310018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:32.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.421+0000 7fd3177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b97a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd3177fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b97a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48942/0 (socket says 172.21.15.83:48942) 2024-04-25T11:43:32.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd3177fe640 1 -- 172.21.15.83:0/10429430 learned_addr learned my addr 172.21.15.83:0/10429430 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd3177fe640 1 -- 172.21.15.83:0/10429430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b975920 msgr2=0x55602b9836c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd3177fe640 1 --2- 172.21.15.83:0/10429430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b975920 0x55602b9836c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd3177fe640 1 -- 172.21.15.83:0/10429430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b974f60 msgr2=0x55602b97a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd3177fe640 1 --2- 172.21.15.83:0/10429430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b974f60 0x55602b97a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd3177fe640 1 -- 172.21.15.83:0/10429430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55602b983c00 con 0x55602b973d90 2024-04-25T11:43:32.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd3177fe640 1 --2- 172.21.15.83:0/10429430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b97a4a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd310018040 tx=0x7fd310020ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1958e84b529223fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd2faff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:32.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd2faff5640 1 -- 172.21.15.83:0/10429430 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31001a070 con 0x55602b973d90 2024-04-25T11:43:32.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd2faff5640 1 -- 172.21.15.83:0/10429430 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fd310015c80 con 0x55602b973d90 2024-04-25T11:43:32.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd32046f740 1 -- 172.21.15.83:0/10429430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd30c01b030 con 0x55602b973d90 2024-04-25T11:43:32.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd2faff5640 1 -- 172.21.15.83:0/10429430 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd310015720 con 0x55602b973d90 2024-04-25T11:43:32.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd32046f740 10 client.15318 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:32.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd32046f740 1 -- 172.21.15.83:0/10429430 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55602b984f00 con 0x55602b973d90 2024-04-25T11:43:32.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd2faff5640 1 -- 172.21.15.83:0/10429430 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd30c01b030 con 0x55602b973d90 2024-04-25T11:43:32.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd2faff5640 1 -- 172.21.15.83:0/10429430 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fd310029980 con 0x55602b973d90 2024-04-25T11:43:32.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd32046f740 10 client.15318 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:32.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd32046f740 10 client.15318 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T11:43:32.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd32046f740 20 client.15318 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:32.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd32046f740 1 --2- 172.21.15.83:0/10429430 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55602b9f5640 0x55602b9f4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd32046f740 4 client.15318 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:43:32.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.422+0000 7fd32046f740 1 -- 172.21.15.83:0/10429430 --> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] -- command(tid 0: {"prefix": "status"}) -- 0x55602b8de4e0 con 0x55602b9f5640 2024-04-25T11:43:32.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.423+0000 7fd3167fc640 1 --2- 172.21.15.83:0/10429430 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55602b9f5640 0x55602b9f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.423+0000 7fd2faff5640 10 client.15318 ms_handle_connect on v2:172.21.15.111:6838/2233531151 2024-04-25T11:43:32.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.423+0000 7fd3167fc640 1 --2- 172.21.15.83:0/10429430 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55602b9f5640 0x55602b9f4350 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=78f509f6daf00ab7 server_cookie=3e47f2b8d20197a9 in_seq=0 out_seq=0 2024-04-25T11:43:32.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.423+0000 7fd2faff5640 1 -- 172.21.15.83:0/10429430 <== mds.1 v2:172.21.15.111:6838/2233531151 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55602b8de4e0 con 0x55602b9f5640 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.423+0000 7fd2faff5640 10 client.15318 handle_command_reply: tid=0 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.423+0000 7fd32046f740 1 client.15318 shutdown 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd32046f740 1 -- 172.21.15.83:0/10429430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 msgr2=0x55602b97a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd32046f740 1 --2- 172.21.15.83:0/10429430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b97a4a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd310018040 tx=0x7fd310020ea0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd32046f740 1 -- 172.21.15.83:0/10429430 shutdown_connections 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd32046f740 1 --2- 172.21.15.83:0/10429430 >> [v2:172.21.15.111:6838/2233531151,v1:172.21.15.111:6839/2233531151] conn(0x55602b9f5640 0x55602b9f4350 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd32046f740 1 --2- 172.21.15.83:0/10429430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55602b975920 0x55602b9836c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd32046f740 1 --2- 172.21.15.83:0/10429430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55602b974f60 0x55602b97a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd32046f740 1 --2- 172.21.15.83:0/10429430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55602b973d90 0x55602b97a4a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd32046f740 1 -- 172.21.15.83:0/10429430 >> 172.21.15.83:0/10429430 conn(0x55602b96d070 msgr2=0x55602b96ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd2faff5640 0 client.15318 ms_handle_reset on v2:172.21.15.111:6838/2233531151 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd32046f740 1 -- 172.21.15.83:0/10429430 shutdown_connections 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd32046f740 1 -- 172.21.15.83:0/10429430 wait complete. 2024-04-25T11:43:32.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd32046f740 20 client.15318 trim_cache size 0 max 16384 2024-04-25T11:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "7fcf54c6-02f7-11ef-bc93-c7b262605968", 2024-04-25T11:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 1, 2024-04-25T11:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24469, 2024-04-25T11:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T11:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T11:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T11:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 104.834598603, 2024-04-25T11:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-04-25T11:43:32.417 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-25T11:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T11:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 120.708480112 2024-04-25T11:43:32.418 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:43:32.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd31d74f640 1 -- 172.21.15.83:0/1426597678 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd3000a3bb0 msgr2=0x7fd3000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd31d74f640 1 --2- 172.21.15.83:0/1426597678 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd3000a3bb0 0x7fd3000a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd3080b9630 tx=0x7fd3080f6040 comp rx=0 tx=0).stop 2024-04-25T11:43:32.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd31d74f640 1 -- 172.21.15.83:0/1426597678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318100f30 msgr2=0x7fd3181971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd31d74f640 1 --2- 172.21.15.83:0/1426597678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318100f30 0x7fd3181971f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd308031500 tx=0x7fd3080043c0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd31d74f640 1 -- 172.21.15.83:0/1426597678 shutdown_connections 2024-04-25T11:43:32.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd31d74f640 1 --2- 172.21.15.83:0/1426597678 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fd3000a3bb0 0x7fd3000a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd31d74f640 1 --2- 172.21.15.83:0/1426597678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd318102a40 0x7fd31819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd31d74f640 1 --2- 172.21.15.83:0/1426597678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd318102080 0x7fd318197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd31d74f640 1 --2- 172.21.15.83:0/1426597678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd318100f30 0x7fd3181971f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd31d74f640 1 -- 172.21.15.83:0/1426597678 >> 172.21.15.83:0/1426597678 conn(0x7fd3180fc4a0 msgr2=0x7fd318105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd31d74f640 1 -- 172.21.15.83:0/1426597678 shutdown_connections 2024-04-25T11:43:32.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.424+0000 7fd31d74f640 1 -- 172.21.15.83:0/1426597678 wait complete. 2024-04-25T11:43:32.421 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T11:43:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.533+0000 7f81eff9b640 1 -- 172.21.15.83:0/4075444663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e8101fe0 msgr2=0x7f81e8102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.533+0000 7f81eff9b640 1 --2- 172.21.15.83:0/4075444663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e8101fe0 0x7f81e8102460 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009900 tx=0x7f81dc02a410 comp rx=0 tx=0).stop 2024-04-25T11:43:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.533+0000 7f81eff9b640 1 -- 172.21.15.83:0/4075444663 shutdown_connections 2024-04-25T11:43:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.533+0000 7f81eff9b640 1 --2- 172.21.15.83:0/4075444663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e8101fe0 0x7f81e8102460 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.533+0000 7f81eff9b640 1 --2- 172.21.15.83:0/4075444663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e81029a0 0x7f81e81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.533+0000 7f81eff9b640 1 --2- 172.21.15.83:0/4075444663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81e8100e90 0x7f81e81012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.533+0000 7f81eff9b640 1 -- 172.21.15.83:0/4075444663 >> 172.21.15.83:0/4075444663 conn(0x7f81e80fc400 msgr2=0x7f81e80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.533+0000 7f81eff9b640 1 -- 172.21.15.83:0/4075444663 shutdown_connections 2024-04-25T11:43:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.533+0000 7f81eff9b640 1 -- 172.21.15.83:0/4075444663 wait complete. 2024-04-25T11:43:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.534+0000 7f81eff9b640 1 Processor -- start 2024-04-25T11:43:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.534+0000 7f81eff9b640 1 -- start start 2024-04-25T11:43:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.534+0000 7f81eff9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81e8100e90 0x7f81e81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.534+0000 7f81eff9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e8101fe0 0x7f81e81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.534+0000 7f81eff9b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e81029a0 0x7f81e819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.534+0000 7f81eff9b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81e810ca60 con 0x7f81e8101fe0 2024-04-25T11:43:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.534+0000 7f81eff9b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81e810c8e0 con 0x7f81e8100e90 2024-04-25T11:43:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.534+0000 7f81eff9b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f81e810cbe0 con 0x7f81e81029a0 2024-04-25T11:43:32.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.534+0000 7f81ed536640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e8101fe0 0x7f81e81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.534+0000 7f81ed536640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e8101fe0 0x7f81e81987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46486/0 (socket says 172.21.15.83:46486) 2024-04-25T11:43:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.534+0000 7f81ed536640 1 -- 172.21.15.83:0/3498897329 learned_addr learned my addr 172.21.15.83:0/3498897329 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.534+0000 7f81edd37640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81e8100e90 0x7f81e81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.534+0000 7f81ee538640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e81029a0 0x7f81e819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.534+0000 7f81ed536640 1 -- 172.21.15.83:0/3498897329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e81029a0 msgr2=0x7f81e819f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.534+0000 7f81ed536640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e81029a0 0x7f81e819f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.535+0000 7f81ed536640 1 -- 172.21.15.83:0/3498897329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81e8100e90 msgr2=0x7f81e81982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.535+0000 7f81ed536640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81e8100e90 0x7f81e81982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.535+0000 7f81ed536640 1 -- 172.21.15.83:0/3498897329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e819fec0 con 0x7f81e8101fe0 2024-04-25T11:43:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.535+0000 7f81ee538640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e81029a0 0x7f81e819f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.535+0000 7f81edd37640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81e8100e90 0x7f81e81982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.535+0000 7f81ed536640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e8101fe0 0x7f81e81987e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f81dc006010 tx=0x7f81dc004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.535+0000 7f81e6ffd640 1 -- 172.21.15.83:0/3498897329 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc03f030 con 0x7f81e8101fe0 2024-04-25T11:43:32.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.535+0000 7f81e6ffd640 1 -- 172.21.15.83:0/3498897329 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81dc003cb0 con 0x7f81e8101fe0 2024-04-25T11:43:32.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.535+0000 7f81eff9b640 1 -- 172.21.15.83:0/3498897329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e81a0150 con 0x7f81e8101fe0 2024-04-25T11:43:32.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.535+0000 7f81e6ffd640 1 -- 172.21.15.83:0/3498897329 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc035070 con 0x7f81e8101fe0 2024-04-25T11:43:32.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.535+0000 7f81eff9b640 1 -- 172.21.15.83:0/3498897329 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e81a06c0 con 0x7f81e8101fe0 2024-04-25T11:43:32.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:32 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/835315641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:43:32.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:32 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/915005318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:43:32.526 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:32 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2148792051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:43:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.537+0000 7f81e6ffd640 1 -- 172.21.15.83:0/3498897329 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f81dc042030 con 0x7f81e8101fe0 2024-04-25T11:43:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.537+0000 7f81f2d8c740 1 Processor -- start 2024-04-25T11:43:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81f2d8c740 1 -- start start 2024-04-25T11:43:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81e6ffd640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f81d00a3bb0 0x7f81d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81f2d8c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d536f90 0x55567d5373b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81f2d8c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d5385e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81f2d8c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d538b20 0x55567d53f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81f2d8c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55567d497c50 con 0x55567d536f90 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81f2d8c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55567d434830 con 0x55567d538160 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81f2d8c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55567d493c10 con 0x55567d538b20 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81ee538640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f81d00a3bb0 0x7f81d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81edd37640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d536f90 0x55567d5373b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81e6ffd640 1 -- 172.21.15.83:0/3498897329 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f81dc0f0ba0 con 0x7f81e8101fe0 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81edd37640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d536f90 0x55567d5373b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46502/0 (socket says 172.21.15.83:46502) 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81edd37640 1 -- 172.21.15.83:0/3456476972 learned_addr learned my addr 172.21.15.83:0/3456476972 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81ee538640 1 --2- 172.21.15.83:0/3456476972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d538b20 0x55567d53f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81edd37640 1 -- 172.21.15.83:0/3456476972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d538b20 msgr2=0x55567d53f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81edd37640 1 --2- 172.21.15.83:0/3456476972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d538b20 0x55567d53f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81ed536640 1 --2- 172.21.15.83:0/3456476972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d5385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81edd37640 1 -- 172.21.15.83:0/3456476972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 msgr2=0x55567d5385e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81edd37640 1 --2- 172.21.15.83:0/3456476972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d5385e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81edd37640 1 -- 172.21.15.83:0/3456476972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55567d1aa2d0 con 0x55567d536f90 2024-04-25T11:43:32.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81ed536640 1 --2- 172.21.15.83:0/3456476972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d5385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:32.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81ee538640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f81d00a3bb0 0x7f81d00a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f81d80126a0 tx=0x7f81d800b970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81edd37640 1 --2- 172.21.15.83:0/3456476972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d536f90 0x55567d5373b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f81e000c4a0 tx=0x7f81e000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ecdbdd16fc5a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81cffff640 1 -- 172.21.15.83:0/3456476972 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e001a070 con 0x55567d536f90 2024-04-25T11:43:32.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81cffff640 1 -- 172.21.15.83:0/3456476972 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e00158c0 con 0x55567d536f90 2024-04-25T11:43:32.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.538+0000 7f81cffff640 1 -- 172.21.15.83:0/3456476972 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e000a7c0 con 0x55567d536f90 2024-04-25T11:43:32.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.539+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3456476972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d536f90 msgr2=0x55567d5373b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.539+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3456476972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d536f90 0x55567d5373b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f81e000c4a0 tx=0x7f81e000c8a0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.539+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3456476972 shutdown_connections 2024-04-25T11:43:32.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.539+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3456476972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d538b20 0x55567d53f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.539+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3456476972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d5385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.539+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3456476972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d536f90 0x55567d5373b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.539+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3456476972 >> 172.21.15.83:0/3456476972 conn(0x55567d52e070 msgr2=0x55567d52e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.539+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3456476972 shutdown_connections 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.539+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3456476972 wait complete. 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81f2d8c740 1 -- start start 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81f2d8c740 1 Processor -- start 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81f2d8c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d536f90 0x55567d541440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81f2d8c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d541980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81f2d8c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d538b20 0x55567d5489c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81f2d8c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55567d53fb30 con 0x55567d538b20 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81f2d8c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55567d432470 con 0x55567d538160 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81f2d8c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55567d53fcb0 con 0x55567d536f90 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81f2d8c740 10 client.0 _ll_register_callbacks cb 0x55567d457f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:32.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81ee538640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d538b20 0x55567d5489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81ee538640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d538b20 0x55567d5489c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46518/0 (socket says 172.21.15.83:46518) 2024-04-25T11:43:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81ee538640 1 -- 172.21.15.83:0/3766027067 learned_addr learned my addr 172.21.15.83:0/3766027067 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81ed536640 1 --2- 172.21.15.83:0/3766027067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d541980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81edd37640 1 --2- 172.21.15.83:0/3766027067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d536f90 0x55567d541440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81ee538640 1 -- 172.21.15.83:0/3766027067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d536f90 msgr2=0x55567d541440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81ee538640 1 --2- 172.21.15.83:0/3766027067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d536f90 0x55567d541440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81ee538640 1 -- 172.21.15.83:0/3766027067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 msgr2=0x55567d541980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81ee538640 1 --2- 172.21.15.83:0/3766027067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d541980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81ee538640 1 -- 172.21.15.83:0/3766027067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55567d2746b0 con 0x55567d538b20 2024-04-25T11:43:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81ee538640 1 --2- 172.21.15.83:0/3766027067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d538b20 0x55567d5489c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f81d800b370 tx=0x7f81d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c63d2e8c9c89120 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.540+0000 7f81cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.541+0000 7f81cd7fa640 1 -- 172.21.15.83:0/3766027067 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8023070 con 0x55567d538b20 2024-04-25T11:43:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.541+0000 7f81cd7fa640 1 -- 172.21.15.83:0/3766027067 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f81d801f800 con 0x55567d538b20 2024-04-25T11:43:32.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.541+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3766027067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81e00158c0 con 0x55567d538b20 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.541+0000 7f81cd7fa640 1 -- 172.21.15.83:0/3766027067 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f81d801fa00 con 0x55567d538b20 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.541+0000 7f81f2d8c740 10 client.15330 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.541+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3766027067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55567d54bd10 con 0x55567d538b20 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.541+0000 7f81cd7fa640 1 -- 172.21.15.83:0/3766027067 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e00158c0 con 0x55567d538b20 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.541+0000 7f81cd7fa640 1 -- 172.21.15.83:0/3766027067 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f81d801b040 con 0x55567d538b20 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.541+0000 7f81f2d8c740 10 client.15330 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.541+0000 7f81f2d8c740 10 client.15330 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.541+0000 7f81f2d8c740 20 client.15330 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.541+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3766027067 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x55567d54e420 0x55567d54f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.541+0000 7f81f2d8c740 4 client.15330 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.541+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3766027067 --> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55567d4a5670 con 0x55567d54e420 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.542+0000 7f81edd37640 1 --2- 172.21.15.83:0/3766027067 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x55567d54e420 0x55567d54f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.542+0000 7f81edd37640 1 --2- 172.21.15.83:0/3766027067 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x55567d54e420 0x55567d54f890 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1356c9a7007eba8 server_cookie=42b046fefb4a4db7 in_seq=0 out_seq=0 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.542+0000 7f81cd7fa640 10 client.15330 ms_handle_connect on v2:172.21.15.94:6838/1461078471 2024-04-25T11:43:32.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.543+0000 7f81cd7fa640 1 -- 172.21.15.83:0/3766027067 <== mds.0 v2:172.21.15.94:6838/1461078471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55567d4a5670 con 0x55567d54e420 2024-04-25T11:43:32.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.543+0000 7f81cd7fa640 10 client.15330 handle_command_reply: tid=0 2024-04-25T11:43:32.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.543+0000 7f81f2d8c740 1 client.15330 shutdown 2024-04-25T11:43:32.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.543+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3766027067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d538b20 msgr2=0x55567d5489c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.543+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3766027067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d538b20 0x55567d5489c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f81d800b370 tx=0x7f81d8004730 comp rx=0 tx=0).stop 2024-04-25T11:43:32.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.543+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3766027067 shutdown_connections 2024-04-25T11:43:32.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.543+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3766027067 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x55567d54e420 0x55567d54f890 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.543+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3766027067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d538b20 0x55567d5489c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.543+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3766027067 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d541980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.543+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3766027067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d536f90 0x55567d541440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.543+0000 7f81cd7fa640 0 client.15330 ms_handle_reset on v2:172.21.15.94:6838/1461078471 2024-04-25T11:43:32.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.543+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3766027067 >> 172.21.15.83:0/3766027067 conn(0x55567d52e070 msgr2=0x55567d52d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.543+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3766027067 shutdown_connections 2024-04-25T11:43:32.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.543+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3766027067 wait complete. 2024-04-25T11:43:32.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.543+0000 7f81f2d8c740 20 client.15330 trim_cache size 0 max 16384 2024-04-25T11:43:32.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.561+0000 7f81f2d8c740 1 Processor -- start 2024-04-25T11:43:32.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.561+0000 7f81f2d8c740 1 -- start start 2024-04-25T11:43:32.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.561+0000 7f81f2d8c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d536f90 0x55567d53b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.561+0000 7f81f2d8c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d548980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.561+0000 7f81f2d8c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d538b20 0x55567d54d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.561+0000 7f81f2d8c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55567d54dbd0 con 0x55567d538b20 2024-04-25T11:43:32.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.561+0000 7f81f2d8c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55567d54e2a0 con 0x55567d538160 2024-04-25T11:43:32.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.561+0000 7f81f2d8c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55567d54e970 con 0x55567d536f90 2024-04-25T11:43:32.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.561+0000 7f81ee538640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d538b20 0x55567d54d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.561+0000 7f81ee538640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d538b20 0x55567d54d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46534/0 (socket says 172.21.15.83:46534) 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.561+0000 7f81ee538640 1 -- 172.21.15.83:0/142837164 learned_addr learned my addr 172.21.15.83:0/142837164 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.562+0000 7f81edd37640 1 --2- 172.21.15.83:0/142837164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d536f90 0x55567d53b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.562+0000 7f81ee538640 1 -- 172.21.15.83:0/142837164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d536f90 msgr2=0x55567d53b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.562+0000 7f81ee538640 1 --2- 172.21.15.83:0/142837164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d536f90 0x55567d53b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.562+0000 7f81ee538640 1 -- 172.21.15.83:0/142837164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 msgr2=0x55567d548980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.562+0000 7f81ee538640 1 --2- 172.21.15.83:0/142837164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d548980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.562+0000 7f81ee538640 1 -- 172.21.15.83:0/142837164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55567d537be0 con 0x55567d538b20 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.562+0000 7f81ee538640 1 --2- 172.21.15.83:0/142837164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d538b20 0x55567d54d1d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f81d8000c00 tx=0x7f81d800fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9ad7aa0121c7ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.562+0000 7f81cf7fe640 1 -- 172.21.15.83:0/142837164 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8023040 con 0x55567d538b20 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.562+0000 7f81cf7fe640 1 -- 172.21.15.83:0/142837164 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81d8037070 con 0x55567d538b20 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.562+0000 7f81cf7fe640 1 -- 172.21.15.83:0/142837164 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d8028460 con 0x55567d538b20 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 -- 172.21.15.83:0/142837164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d538b20 msgr2=0x55567d54d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/142837164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d538b20 0x55567d54d1d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f81d8000c00 tx=0x7f81d800fc30 comp rx=0 tx=0).stop 2024-04-25T11:43:32.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 -- 172.21.15.83:0/142837164 shutdown_connections 2024-04-25T11:43:32.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/142837164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d538b20 0x55567d54d1d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/142837164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d548980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/142837164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d536f90 0x55567d53b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 -- 172.21.15.83:0/142837164 >> 172.21.15.83:0/142837164 conn(0x55567d52e070 msgr2=0x55567d5290b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 -- 172.21.15.83:0/142837164 shutdown_connections 2024-04-25T11:43:32.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 -- 172.21.15.83:0/142837164 wait complete. 2024-04-25T11:43:32.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 -- start start 2024-04-25T11:43:32.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 Processor -- start 2024-04-25T11:43:32.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d536f90 0x55567d543910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d543e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d538b20 0x55567d5b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55567d54f710 con 0x55567d536f90 2024-04-25T11:43:32.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55567d54f040 con 0x55567d538160 2024-04-25T11:43:32.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55567d54fde0 con 0x55567d538b20 2024-04-25T11:43:32.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81f2d8c740 10 client.0 _ll_register_callbacks cb 0x7f81d800bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:43:32.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81edd37640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d536f90 0x55567d543910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81edd37640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d536f90 0x55567d543910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46546/0 (socket says 172.21.15.83:46546) 2024-04-25T11:43:32.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81edd37640 1 -- 172.21.15.83:0/3752019634 learned_addr learned my addr 172.21.15.83:0/3752019634 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.563+0000 7f81ee538640 1 --2- 172.21.15.83:0/3752019634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d538b20 0x55567d5b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81ed536640 1 --2- 172.21.15.83:0/3752019634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d543e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81edd37640 1 -- 172.21.15.83:0/3752019634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d538b20 msgr2=0x55567d5b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81edd37640 1 --2- 172.21.15.83:0/3752019634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d538b20 0x55567d5b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81edd37640 1 -- 172.21.15.83:0/3752019634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 msgr2=0x55567d543e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:43:32.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81edd37640 1 --2- 172.21.15.83:0/3752019634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d543e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81edd37640 1 -- 172.21.15.83:0/3752019634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55567d5b2930 con 0x55567d536f90 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81ed536640 1 --2- 172.21.15.83:0/3752019634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d543e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81ee538640 1 --2- 172.21.15.83:0/3752019634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d538b20 0x55567d5b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81edd37640 1 --2- 172.21.15.83:0/3752019634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d536f90 0x55567d543910 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009440 tx=0x7f81e0009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df496ec81eee14fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81cd7fa640 1 -- 172.21.15.83:0/3752019634 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e001a040 con 0x55567d536f90 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81cd7fa640 1 -- 172.21.15.83:0/3752019634 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f81e0025160 con 0x55567d536f90 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3752019634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81d8037070 con 0x55567d536f90 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81cd7fa640 1 -- 172.21.15.83:0/3752019634 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f81e0025360 con 0x55567d536f90 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81f2d8c740 10 client.15336 fetch_fsmap learned FSMap version 24 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3752019634 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55567d5b3ce0 con 0x55567d536f90 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81cd7fa640 1 -- 172.21.15.83:0/3752019634 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81d8037070 con 0x55567d536f90 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81cd7fa640 1 -- 172.21.15.83:0/3752019634 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f81e0015c80 con 0x55567d536f90 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81f2d8c740 10 client.15336 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T11:43:32.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81f2d8c740 10 client.15336 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:43:32.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81f2d8c740 20 client.15336 populate_metadata read hostname 'smithi083' 2024-04-25T11:43:32.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3752019634 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x55567d5b84e0 0x55567d5b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81f2d8c740 4 client.15336 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:43:32.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.564+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3752019634 --> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] -- command(tid 0: {"prefix": "status"}) -- 0x55567d49f4e0 con 0x55567d5b84e0 2024-04-25T11:43:32.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81ed536640 1 --2- 172.21.15.83:0/3752019634 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x55567d5b84e0 0x55567d5b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81ed536640 1 --2- 172.21.15.83:0/3752019634 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x55567d5b84e0 0x55567d5b71f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab3d0e993e214010 server_cookie=cb3bc326d426bf3b in_seq=0 out_seq=0 2024-04-25T11:43:32.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81cd7fa640 10 client.15336 ms_handle_connect on v2:172.21.15.94:6838/1461078471 2024-04-25T11:43:32.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81cd7fa640 1 -- 172.21.15.83:0/3752019634 <== mds.0 v2:172.21.15.94:6838/1461078471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55567d49f4e0 con 0x55567d5b84e0 2024-04-25T11:43:32.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81cd7fa640 10 client.15336 handle_command_reply: tid=0 2024-04-25T11:43:32.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81f2d8c740 1 client.15336 shutdown 2024-04-25T11:43:32.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3752019634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d536f90 msgr2=0x55567d543910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3752019634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d536f90 0x55567d543910 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009440 tx=0x7f81e0009ba0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3752019634 shutdown_connections 2024-04-25T11:43:32.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3752019634 >> [v2:172.21.15.94:6838/1461078471,v1:172.21.15.94:6839/1461078471] conn(0x55567d5b84e0 0x55567d5b71f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3752019634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55567d538b20 0x55567d5b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3752019634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55567d538160 0x55567d543e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81cd7fa640 0 client.15336 ms_handle_reset on v2:172.21.15.94:6838/1461078471 2024-04-25T11:43:32.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81f2d8c740 1 --2- 172.21.15.83:0/3752019634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55567d536f90 0x55567d543910 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3752019634 >> 172.21.15.83:0/3752019634 conn(0x55567d52e070 msgr2=0x55567d5290b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3752019634 shutdown_connections 2024-04-25T11:43:32.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.565+0000 7f81f2d8c740 1 -- 172.21.15.83:0/3752019634 wait complete. 2024-04-25T11:43:32.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.566+0000 7f81f2d8c740 20 client.15336 trim_cache size 0 max 16384 2024-04-25T11:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:43:32.559 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "7fcf54c6-02f7-11ef-bc93-c7b262605968", 2024-04-25T11:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-25T11:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24479, 2024-04-25T11:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-25T11:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-25T11:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-25T11:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 107.12623177099999, 2024-04-25T11:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 24, 2024-04-25T11:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-25T11:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T11:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 112.494091553 2024-04-25T11:43:32.560 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:43:32.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.566+0000 7f81eff9b640 1 -- 172.21.15.83:0/3498897329 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f81d00a3bb0 msgr2=0x7f81d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.566+0000 7f81eff9b640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f81d00a3bb0 0x7f81d00a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f81d80126a0 tx=0x7f81d800b970 comp rx=0 tx=0).stop 2024-04-25T11:43:32.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.566+0000 7f81eff9b640 1 -- 172.21.15.83:0/3498897329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e8101fe0 msgr2=0x7f81e81987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.566+0000 7f81eff9b640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e8101fe0 0x7f81e81987e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f81dc006010 tx=0x7f81dc004430 comp rx=0 tx=0).stop 2024-04-25T11:43:32.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.566+0000 7f81eff9b640 1 -- 172.21.15.83:0/3498897329 shutdown_connections 2024-04-25T11:43:32.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.566+0000 7f81eff9b640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f81d00a3bb0 0x7f81d00a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.566+0000 7f81eff9b640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81e8101fe0 0x7f81e81987e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.566+0000 7f81eff9b640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81e81029a0 0x7f81e819f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.566+0000 7f81eff9b640 1 --2- 172.21.15.83:0/3498897329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81e8100e90 0x7f81e81982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.566+0000 7f81eff9b640 1 -- 172.21.15.83:0/3498897329 >> 172.21.15.83:0/3498897329 conn(0x7f81e80fc400 msgr2=0x7f81e80690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.566+0000 7f81eff9b640 1 -- 172.21.15.83:0/3498897329 shutdown_connections 2024-04-25T11:43:32.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.566+0000 7f81eff9b640 1 -- 172.21.15.83:0/3498897329 wait complete. 2024-04-25T11:43:32.563 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T11:43:32.574 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T11:43:32.575 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T11:43:32.575 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T11:43:32.575 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T11:43:32.576 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T11:43:32.576 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T11:43:32.576 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T11:43:32.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.686+0000 7fbc926e8640 1 -- 172.21.15.83:0/2653960174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c103b70 msgr2=0x7fbc8c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.686+0000 7fbc926e8640 1 --2- 172.21.15.83:0/2653960174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c103b70 0x7fbc8c10a400 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009ca0 tx=0x7fbc8402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.686+0000 7fbc926e8640 1 -- 172.21.15.83:0/2653960174 shutdown_connections 2024-04-25T11:43:32.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.686+0000 7fbc926e8640 1 --2- 172.21.15.83:0/2653960174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c103b70 0x7fbc8c10a400 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.686+0000 7fbc926e8640 1 --2- 172.21.15.83:0/2653960174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc8c1031b0 0x7fbc8c103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.686+0000 7fbc926e8640 1 --2- 172.21.15.83:0/2653960174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc8c101fc0 0x7fbc8c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.686+0000 7fbc926e8640 1 -- 172.21.15.83:0/2653960174 >> 172.21.15.83:0/2653960174 conn(0x7fbc8c0fd530 msgr2=0x7fbc8c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:32.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.686+0000 7fbc926e8640 1 -- 172.21.15.83:0/2653960174 shutdown_connections 2024-04-25T11:43:32.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.686+0000 7fbc926e8640 1 -- 172.21.15.83:0/2653960174 wait complete. 2024-04-25T11:43:32.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.687+0000 7fbc926e8640 1 Processor -- start 2024-04-25T11:43:32.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.687+0000 7fbc926e8640 1 -- start start 2024-04-25T11:43:32.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.687+0000 7fbc926e8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc8c101fc0 0x7fbc8c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.687+0000 7fbc926e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc8c1031b0 0x7fbc8c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.687+0000 7fbc926e8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c103b70 0x7fbc8c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.687+0000 7fbc926e8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c10db00 con 0x7fbc8c103b70 2024-04-25T11:43:32.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.687+0000 7fbc926e8640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c10d980 con 0x7fbc8c1031b0 2024-04-25T11:43:32.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.687+0000 7fbc926e8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8c10dc80 con 0x7fbc8c101fc0 2024-04-25T11:43:32.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.687+0000 7fbc90c85640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c103b70 0x7fbc8c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.687+0000 7fbc90c85640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c103b70 0x7fbc8c19f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46570/0 (socket says 172.21.15.83:46570) 2024-04-25T11:43:32.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.687+0000 7fbc90c85640 1 -- 172.21.15.83:0/1695614303 learned_addr learned my addr 172.21.15.83:0/1695614303 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.687+0000 7fbc8bfff640 1 --2- 172.21.15.83:0/1695614303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc8c101fc0 0x7fbc8c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.687+0000 7fbc8b7fe640 1 --2- 172.21.15.83:0/1695614303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc8c1031b0 0x7fbc8c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.688+0000 7fbc90c85640 1 -- 172.21.15.83:0/1695614303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc8c101fc0 msgr2=0x7fbc8c1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.688+0000 7fbc90c85640 1 --2- 172.21.15.83:0/1695614303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc8c101fc0 0x7fbc8c1980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.688+0000 7fbc90c85640 1 -- 172.21.15.83:0/1695614303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc8c1031b0 msgr2=0x7fbc8c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.688+0000 7fbc90c85640 1 --2- 172.21.15.83:0/1695614303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc8c1031b0 0x7fbc8c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.688+0000 7fbc90c85640 1 -- 172.21.15.83:0/1695614303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8c19fd10 con 0x7fbc8c103b70 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.688+0000 7fbc90c85640 1 --2- 172.21.15.83:0/1695614303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c103b70 0x7fbc8c19f6a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbc840097b0 tx=0x7fbc84003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.688+0000 7fbc897fa640 1 -- 172.21.15.83:0/1695614303 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84004240 con 0x7fbc8c103b70 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.688+0000 7fbc897fa640 1 -- 172.21.15.83:0/1695614303 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc840317c0 con 0x7fbc8c103b70 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.688+0000 7fbc897fa640 1 -- 172.21.15.83:0/1695614303 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8402fd80 con 0x7fbc8c103b70 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.688+0000 7fbc926e8640 1 -- 172.21.15.83:0/1695614303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8c19ffa0 con 0x7fbc8c103b70 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.688+0000 7fbc926e8640 1 -- 172.21.15.83:0/1695614303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8c1a0460 con 0x7fbc8c103b70 2024-04-25T11:43:32.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.689+0000 7fbc926e8640 1 -- 172.21.15.83:0/1695614303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8c1092e0 con 0x7fbc8c103b70 2024-04-25T11:43:32.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.690+0000 7fbc897fa640 1 -- 172.21.15.83:0/1695614303 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbc840043e0 con 0x7fbc8c103b70 2024-04-25T11:43:32.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.690+0000 7fbc897fa640 1 --2- 172.21.15.83:0/1695614303 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbc740a3c50 0x7fbc740a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:32.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.691+0000 7fbc8bfff640 1 --2- 172.21.15.83:0/1695614303 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbc740a3c50 0x7fbc740a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.691+0000 7fbc897fa640 1 -- 172.21.15.83:0/1695614303 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fbc840f1e40 con 0x7fbc8c103b70 2024-04-25T11:43:32.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.691+0000 7fbc8bfff640 1 --2- 172.21.15.83:0/1695614303 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbc740a3c50 0x7fbc740a6110 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0058c0 tx=0x7fbc7c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:32.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.695+0000 7fbc897fa640 1 -- 172.21.15.83:0/1695614303 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc840be950 con 0x7fbc8c103b70 2024-04-25T11:43:32.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:32.923+0000 7fbc926e8640 1 -- 172.21.15.83:0/1695614303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fbc8c062bf0 con 0x7fbc8c103b70 2024-04-25T11:43:33.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.061+0000 7fbc897fa640 1 -- 172.21.15.83:0/1695614303 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fbc840098b0 con 0x7fbc8c103b70 2024-04-25T11:43:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.063+0000 7fbc926e8640 1 -- 172.21.15.83:0/1695614303 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbc740a3c50 msgr2=0x7fbc740a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.063+0000 7fbc926e8640 1 --2- 172.21.15.83:0/1695614303 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbc740a3c50 0x7fbc740a6110 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0058c0 tx=0x7fbc7c00a4c0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.063+0000 7fbc926e8640 1 -- 172.21.15.83:0/1695614303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c103b70 msgr2=0x7fbc8c19f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.063+0000 7fbc926e8640 1 --2- 172.21.15.83:0/1695614303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c103b70 0x7fbc8c19f6a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbc840097b0 tx=0x7fbc84003c20 comp rx=0 tx=0).stop 2024-04-25T11:43:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.063+0000 7fbc926e8640 1 -- 172.21.15.83:0/1695614303 shutdown_connections 2024-04-25T11:43:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.063+0000 7fbc926e8640 1 --2- 172.21.15.83:0/1695614303 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fbc740a3c50 0x7fbc740a6110 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.063+0000 7fbc926e8640 1 --2- 172.21.15.83:0/1695614303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc8c103b70 0x7fbc8c19f6a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.063+0000 7fbc926e8640 1 --2- 172.21.15.83:0/1695614303 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc8c1031b0 0x7fbc8c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.063+0000 7fbc926e8640 1 --2- 172.21.15.83:0/1695614303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc8c101fc0 0x7fbc8c1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.063+0000 7fbc926e8640 1 -- 172.21.15.83:0/1695614303 >> 172.21.15.83:0/1695614303 conn(0x7fbc8c0fd530 msgr2=0x7fbc8c1064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.064+0000 7fbc926e8640 1 -- 172.21.15.83:0/1695614303 shutdown_connections 2024-04-25T11:43:33.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.064+0000 7fbc926e8640 1 -- 172.21.15.83:0/1695614303 wait complete. 2024-04-25T11:43:33.070 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T11:43:33.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.188+0000 7f97a576f640 1 -- 172.21.15.83:0/2362652414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a0102080 msgr2=0x7f97a0102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.188+0000 7f97a576f640 1 --2- 172.21.15.83:0/2362652414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a0102080 0x7f97a0102500 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9794009ca0 tx=0x7f979402a8c0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.188+0000 7f97a576f640 1 -- 172.21.15.83:0/2362652414 shutdown_connections 2024-04-25T11:43:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.188+0000 7f97a576f640 1 --2- 172.21.15.83:0/2362652414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a0102080 0x7f97a0102500 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.188+0000 7f97a576f640 1 --2- 172.21.15.83:0/2362652414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97a0102a40 0x7f97a0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.188+0000 7f97a576f640 1 --2- 172.21.15.83:0/2362652414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97a0100ea0 0x7f97a01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.188+0000 7f97a576f640 1 -- 172.21.15.83:0/2362652414 >> 172.21.15.83:0/2362652414 conn(0x7f97a00fc460 msgr2=0x7f97a00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.188+0000 7f97a576f640 1 -- 172.21.15.83:0/2362652414 shutdown_connections 2024-04-25T11:43:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.188+0000 7f97a576f640 1 -- 172.21.15.83:0/2362652414 wait complete. 2024-04-25T11:43:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.189+0000 7f97a576f640 1 Processor -- start 2024-04-25T11:43:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.189+0000 7f97a576f640 1 -- start start 2024-04-25T11:43:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.189+0000 7f97a576f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97a0100ea0 0x7f97a0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:33.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.189+0000 7f97a576f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97a0102080 0x7f97a0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.189+0000 7f97a576f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a0102a40 0x7f97a019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.189+0000 7f97a576f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97a010ca20 con 0x7f97a0102a40 2024-04-25T11:43:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.189+0000 7f97a576f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f97a010c8a0 con 0x7f97a0102080 2024-04-25T11:43:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.189+0000 7f97a576f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f97a010cba0 con 0x7f97a0100ea0 2024-04-25T11:43:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.189+0000 7f979f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a0102a40 0x7f97a019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.189+0000 7f979f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a0102a40 0x7f97a019f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46606/0 (socket says 172.21.15.83:46606) 2024-04-25T11:43:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.189+0000 7f979f7fe640 1 -- 172.21.15.83:0/3061576983 learned_addr learned my addr 172.21.15.83:0/3061576983 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.189+0000 7f979effd640 1 --2- 172.21.15.83:0/3061576983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97a0100ea0 0x7f97a0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.189+0000 7f979e7fc640 1 --2- 172.21.15.83:0/3061576983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97a0102080 0x7f97a0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.190+0000 7f979f7fe640 1 -- 172.21.15.83:0/3061576983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97a0100ea0 msgr2=0x7f97a0198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.190+0000 7f979f7fe640 1 --2- 172.21.15.83:0/3061576983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97a0100ea0 0x7f97a0198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.190+0000 7f979f7fe640 1 -- 172.21.15.83:0/3061576983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97a0102080 msgr2=0x7f97a0198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:33.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.190+0000 7f979f7fe640 1 --2- 172.21.15.83:0/3061576983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97a0102080 0x7f97a0198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.190+0000 7f979f7fe640 1 -- 172.21.15.83:0/3061576983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a019fd20 con 0x7f97a0102a40 2024-04-25T11:43:33.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.190+0000 7f979f7fe640 1 --2- 172.21.15.83:0/3061576983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a0102a40 0x7f97a019f6b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f979000e970 tx=0x7f979000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:33.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.190+0000 7f9787fff640 1 -- 172.21.15.83:0/3061576983 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979000ccc0 con 0x7f97a0102a40 2024-04-25T11:43:33.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.190+0000 7f9787fff640 1 -- 172.21.15.83:0/3061576983 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f979000ce60 con 0x7f97a0102a40 2024-04-25T11:43:33.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.190+0000 7f97a576f640 1 -- 172.21.15.83:0/3061576983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a019ffb0 con 0x7f97a0102a40 2024-04-25T11:43:33.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.190+0000 7f9787fff640 1 -- 172.21.15.83:0/3061576983 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790010430 con 0x7f97a0102a40 2024-04-25T11:43:33.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.190+0000 7f97a576f640 1 -- 172.21.15.83:0/3061576983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a01a0480 con 0x7f97a0102a40 2024-04-25T11:43:33.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.191+0000 7f97a576f640 1 -- 172.21.15.83:0/3061576983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a004fce0 con 0x7f97a0102a40 2024-04-25T11:43:33.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.192+0000 7f9787fff640 1 -- 172.21.15.83:0/3061576983 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f9790004f90 con 0x7f97a0102a40 2024-04-25T11:43:33.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.193+0000 7f9787fff640 1 --2- 172.21.15.83:0/3061576983 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f97880a3ca0 0x7f97880a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:33.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.193+0000 7f979effd640 1 -- 172.21.15.83:0/3061576983 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f97880a3ca0 msgr2=0x7f97880a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1157946519 2024-04-25T11:43:33.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.193+0000 7f9787fff640 1 -- 172.21.15.83:0/3061576983 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f97900d0980 con 0x7f97a0102a40 2024-04-25T11:43:33.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.193+0000 7f979effd640 1 --2- 172.21.15.83:0/3061576983 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f97880a3ca0 0x7f97880a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:43:33.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.197+0000 7f9787fff640 1 -- 172.21.15.83:0/3061576983 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9790053ff0 con 0x7f97a0102a40 2024-04-25T11:43:33.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:33 smithi111 ceph-mon[32784]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T11:43:33.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:33 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1695614303' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T11:43:33.321 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:33 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: ignoring --setuser ceph since I am not root 2024-04-25T11:43:33.321 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:33 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: ignoring --setgroup ceph since I am not root 2024-04-25T11:43:33.321 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:33 smithi111 ceph-mgr[34631]: -- 172.21.15.111:0/495521222 <== mon.1 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56543b4fe5a0 con 0x56543b4d3000 2024-04-25T11:43:33.321 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:33 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:33.245+0000 7f202e97f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:43:33.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.385+0000 7f97a576f640 1 -- 172.21.15.83:0/3061576983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f97a01a0770 con 0x7f97a0102a40 2024-04-25T11:43:33.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.389+0000 7f9787fff640 1 -- 172.21.15.83:0/3061576983 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7f9790087020 con 0x7f97a0102a40 2024-04-25T11:43:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.391+0000 7f97a576f640 1 -- 172.21.15.83:0/3061576983 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f97880a3ca0 msgr2=0x7f97880a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:43:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.391+0000 7f97a576f640 1 --2- 172.21.15.83:0/3061576983 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f97880a3ca0 0x7f97880a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.391+0000 7f97a576f640 1 -- 172.21.15.83:0/3061576983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a0102a40 msgr2=0x7f97a019f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.391+0000 7f97a576f640 1 --2- 172.21.15.83:0/3061576983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a0102a40 0x7f97a019f6b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f979000e970 tx=0x7f979000ed70 comp rx=0 tx=0).stop 2024-04-25T11:43:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.391+0000 7f97a576f640 1 -- 172.21.15.83:0/3061576983 shutdown_connections 2024-04-25T11:43:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.391+0000 7f97a576f640 1 --2- 172.21.15.83:0/3061576983 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f97880a3ca0 0x7f97880a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.391+0000 7f97a576f640 1 --2- 172.21.15.83:0/3061576983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97a0102a40 0x7f97a019f6b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.391+0000 7f97a576f640 1 --2- 172.21.15.83:0/3061576983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97a0102080 0x7f97a0198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.391+0000 7f97a576f640 1 --2- 172.21.15.83:0/3061576983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97a0100ea0 0x7f97a0198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.391+0000 7f97a576f640 1 -- 172.21.15.83:0/3061576983 >> 172.21.15.83:0/3061576983 conn(0x7f97a00fc460 msgr2=0x7f97a00fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.391+0000 7f97a576f640 1 -- 172.21.15.83:0/3061576983 shutdown_connections 2024-04-25T11:43:33.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.391+0000 7f97a576f640 1 -- 172.21.15.83:0/3061576983 wait complete. 2024-04-25T11:43:33.388 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:33 smithi083 ceph-mon[29330]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T11:43:33.388 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:33 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1695614303' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T11:43:33.388 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:33 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: ignoring --setuser ceph since I am not root 2024-04-25T11:43:33.389 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:33 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: ignoring --setgroup ceph since I am not root 2024-04-25T11:43:33.389 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:33 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:33.250+0000 7efec1198200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:43:33.389 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T11:43:33.410 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:33 smithi094 ceph-mon[33032]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-25T11:43:33.410 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:33 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1695614303' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T11:43:33.411 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: ignoring --setuser ceph since I am not root 2024-04-25T11:43:33.411 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: ignoring --setgroup ceph since I am not root 2024-04-25T11:43:33.411 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:33 smithi094 ceph-mgr[34427]: -- 172.21.15.94:0/1071912895 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640d82c05a0 con 0x5640d8294c00 2024-04-25T11:43:33.411 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:33.241+0000 7f029ad1a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:43:33.411 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:33.423+0000 7f029ad1a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:43:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.505+0000 7f104422a640 1 -- 172.21.15.83:0/3675996174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f103c100e80 msgr2=0x7f103c1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.505+0000 7f104422a640 1 --2- 172.21.15.83:0/3675996174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f103c100e80 0x7f103c1012a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1038009900 tx=0x7f103802a410 comp rx=0 tx=0).stop 2024-04-25T11:43:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.505+0000 7f104422a640 1 -- 172.21.15.83:0/3675996174 shutdown_connections 2024-04-25T11:43:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.505+0000 7f104422a640 1 --2- 172.21.15.83:0/3675996174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f103c102a20 0x7f103c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.505+0000 7f104422a640 1 --2- 172.21.15.83:0/3675996174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f103c102060 0x7f103c1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.505+0000 7f104422a640 1 --2- 172.21.15.83:0/3675996174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f103c100e80 0x7f103c1012a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.505+0000 7f104422a640 1 -- 172.21.15.83:0/3675996174 >> 172.21.15.83:0/3675996174 conn(0x7f103c0fc460 msgr2=0x7f103c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.505+0000 7f104422a640 1 -- 172.21.15.83:0/3675996174 shutdown_connections 2024-04-25T11:43:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.505+0000 7f104422a640 1 -- 172.21.15.83:0/3675996174 wait complete. 2024-04-25T11:43:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.505+0000 7f104422a640 1 Processor -- start 2024-04-25T11:43:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.505+0000 7f104422a640 1 -- start start 2024-04-25T11:43:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.505+0000 7f104422a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f103c100e80 0x7f103c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f104422a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f103c102060 0x7f103c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f104422a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f103c102a20 0x7f103c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f104422a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f103c10ca20 con 0x7f103c102a20 2024-04-25T11:43:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f104422a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f103c10c8a0 con 0x7f103c100e80 2024-04-25T11:43:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f104422a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f103c10cba0 con 0x7f103c102060 2024-04-25T11:43:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f10427c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f103c102a20 0x7f103c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f10427c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f103c102a20 0x7f103c19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46618/0 (socket says 172.21.15.83:46618) 2024-04-25T11:43:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f10427c7640 1 -- 172.21.15.83:0/1895019138 learned_addr learned my addr 172.21.15.83:0/1895019138 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f10417c5640 1 --2- 172.21.15.83:0/1895019138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f103c102060 0x7f103c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f1041fc6640 1 --2- 172.21.15.83:0/1895019138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f103c100e80 0x7f103c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f10427c7640 1 -- 172.21.15.83:0/1895019138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f103c102060 msgr2=0x7f103c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f10427c7640 1 --2- 172.21.15.83:0/1895019138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f103c102060 0x7f103c1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f10427c7640 1 -- 172.21.15.83:0/1895019138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f103c100e80 msgr2=0x7f103c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f10427c7640 1 --2- 172.21.15.83:0/1895019138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f103c100e80 0x7f103c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f10427c7640 1 -- 172.21.15.83:0/1895019138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c19fdb0 con 0x7f103c102a20 2024-04-25T11:43:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f10427c7640 1 --2- 172.21.15.83:0/1895019138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f103c102a20 0x7f103c19f740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f103400eb10 tx=0x7f103400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f1032ffd640 1 -- 172.21.15.83:0/1895019138 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034004b20 con 0x7f103c102a20 2024-04-25T11:43:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f1032ffd640 1 -- 172.21.15.83:0/1895019138 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1034004cc0 con 0x7f103c102a20 2024-04-25T11:43:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f104422a640 1 -- 172.21.15.83:0/1895019138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103c1a00a0 con 0x7f103c102a20 2024-04-25T11:43:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f1032ffd640 1 -- 172.21.15.83:0/1895019138 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034010670 con 0x7f103c102a20 2024-04-25T11:43:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.506+0000 7f104422a640 1 -- 172.21.15.83:0/1895019138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f103c1a05f0 con 0x7f103c102a20 2024-04-25T11:43:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.507+0000 7f104422a640 1 -- 172.21.15.83:0/1895019138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f103c1a08b0 con 0x7f103c102a20 2024-04-25T11:43:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.508+0000 7f1032ffd640 1 -- 172.21.15.83:0/1895019138 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f1034005020 con 0x7f103c102a20 2024-04-25T11:43:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.509+0000 7f1032ffd640 1 --2- 172.21.15.83:0/1895019138 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f10280a3ca0 0x7f10280a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.509+0000 7f1041fc6640 1 -- 172.21.15.83:0/1895019138 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f10280a3ca0 msgr2=0x7f10280a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1157946519 2024-04-25T11:43:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.509+0000 7f1032ffd640 1 -- 172.21.15.83:0/1895019138 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f10340d0970 con 0x7f103c102a20 2024-04-25T11:43:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.509+0000 7f1041fc6640 1 --2- 172.21.15.83:0/1895019138 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f10280a3ca0 0x7f10280a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:43:33.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.514+0000 7f1032ffd640 1 -- 172.21.15.83:0/1895019138 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f103409d360 con 0x7f103c102a20 2024-04-25T11:43:33.654 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:33 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:33.443+0000 7efec1198200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:43:33.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:33 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:33.552+0000 7efec1198200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:43:33.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.699+0000 7f104422a640 1 -- 172.21.15.83:0/1895019138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f103c1a0d10 con 0x7f103c102a20 2024-04-25T11:43:33.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.703+0000 7f1032ffd640 1 -- 172.21.15.83:0/1895019138 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f10340bbe20 con 0x7f103c102a20 2024-04-25T11:43:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.705+0000 7f104422a640 1 -- 172.21.15.83:0/1895019138 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f10280a3ca0 msgr2=0x7f10280a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:43:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.705+0000 7f104422a640 1 --2- 172.21.15.83:0/1895019138 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f10280a3ca0 0x7f10280a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.705+0000 7f104422a640 1 -- 172.21.15.83:0/1895019138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f103c102a20 msgr2=0x7f103c19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.705+0000 7f104422a640 1 --2- 172.21.15.83:0/1895019138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f103c102a20 0x7f103c19f740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f103400eb10 tx=0x7f103400c2d0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.705+0000 7f104422a640 1 -- 172.21.15.83:0/1895019138 shutdown_connections 2024-04-25T11:43:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.705+0000 7f104422a640 1 --2- 172.21.15.83:0/1895019138 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f10280a3ca0 0x7f10280a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.705+0000 7f104422a640 1 --2- 172.21.15.83:0/1895019138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f103c102a20 0x7f103c19f740 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.705+0000 7f104422a640 1 --2- 172.21.15.83:0/1895019138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f103c102060 0x7f103c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.705+0000 7f104422a640 1 --2- 172.21.15.83:0/1895019138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f103c100e80 0x7f103c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:33.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.705+0000 7f104422a640 1 -- 172.21.15.83:0/1895019138 >> 172.21.15.83:0/1895019138 conn(0x7f103c0fc460 msgr2=0x7f103c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.705+0000 7f104422a640 1 -- 172.21.15.83:0/1895019138 shutdown_connections 2024-04-25T11:43:33.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:33.705+0000 7f104422a640 1 -- 172.21.15.83:0/1895019138 wait complete. 2024-04-25T11:43:33.703 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T11:43:33.725 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T11:43:33.725 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T11:43:33.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:33.725 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T11:43:33.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:33.741 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-25T11:43:33.772 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:33.527+0000 7f029ad1a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:43:33.805 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2024-04-25T11:43:33.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:43:33.807 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-25T11:43:33.807 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:43:33.807 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:43:33.807 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T11:43:33.821 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:33 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:33.429+0000 7f202e97f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:43:33.821 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:33 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:33.531+0000 7f202e97f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:43:34.061 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:33 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:33.872+0000 7efec1198200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:43:34.061 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:33 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:33.962+0000 7efec1198200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:43:34.096 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:43:34.272 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:33.835+0000 7f029ad1a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:43:34.272 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:33.920+0000 7f029ad1a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:43:34.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:34 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1695614303' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T11:43:34.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:34 smithi094 ceph-mon[33032]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T11:43:34.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:34 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3061576983' entity='client.admin' 2024-04-25T11:43:34.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:34 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1895019138' entity='client.admin' 2024-04-25T11:43:34.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:34 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1695614303' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T11:43:34.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:34 smithi111 ceph-mon[32784]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T11:43:34.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:34 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3061576983' entity='client.admin' 2024-04-25T11:43:34.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:34 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1895019138' entity='client.admin' 2024-04-25T11:43:34.321 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:33 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:33.840+0000 7f202e97f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:43:34.321 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:33 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:33.925+0000 7f202e97f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:43:34.365 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:34 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1695614303' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T11:43:34.365 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:34 smithi083 ceph-mon[29330]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-25T11:43:34.365 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:34 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3061576983' entity='client.admin' 2024-04-25T11:43:34.366 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:34 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1895019138' entity='client.admin' 2024-04-25T11:43:35.412 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:35 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:35.090+0000 7f029ad1a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:43:35.413 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:35 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:35.259+0000 7f029ad1a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:43:35.418 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:35 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:35.177+0000 7efec1198200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:43:35.418 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:35 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:35.348+0000 7efec1198200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:43:35.419 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:35 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:35.095+0000 7f202e97f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:43:35.419 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:35 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:35.264+0000 7f202e97f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:43:35.419 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:35 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:35.433+0000 7f202e97f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:43:35.700 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:35 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:35.426+0000 7f029ad1a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:43:35.765 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:35 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:35.547+0000 7efec1198200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:43:35.993 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:35 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:35.713+0000 7f029ad1a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:43:35.993 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:35 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:35.806+0000 7f029ad1a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:43:36.005 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:35 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:35.723+0000 7f202e97f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:43:36.005 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:35 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:35.816+0000 7f202e97f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:43:36.005 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:36 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:36.018+0000 7f202e97f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:43:36.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.023+0000 7f30cfeac700 1 -- 172.21.15.83:0/4201854616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30c81022f0 msgr2=0x7f30c810e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.023+0000 7f30cfeac700 1 --2- 172.21.15.83:0/4201854616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30c81022f0 0x7f30c810e7e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f30c000b0a0 tx=0x7f30c000b3b0 comp rx=0 tx=0).stop 2024-04-25T11:43:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.023+0000 7f30cfeac700 1 -- 172.21.15.83:0/4201854616 shutdown_connections 2024-04-25T11:43:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.023+0000 7f30cfeac700 1 --2- 172.21.15.83:0/4201854616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30c81022f0 0x7f30c810e7e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.023+0000 7f30cfeac700 1 --2- 172.21.15.83:0/4201854616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c8107980 0x7f30c8107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.023+0000 7f30cfeac700 1 --2- 172.21.15.83:0/4201854616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30c8101930 0x7f30c8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.023+0000 7f30cfeac700 1 -- 172.21.15.83:0/4201854616 >> 172.21.15.83:0/4201854616 conn(0x7f30c80788b0 msgr2=0x7f30c8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.023+0000 7f30cfeac700 1 -- 172.21.15.83:0/4201854616 shutdown_connections 2024-04-25T11:43:36.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.023+0000 7f30cfeac700 1 -- 172.21.15.83:0/4201854616 wait complete. 2024-04-25T11:43:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.024+0000 7f30cfeac700 1 Processor -- start 2024-04-25T11:43:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.024+0000 7f30cfeac700 1 -- start start 2024-04-25T11:43:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.024+0000 7f30cfeac700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30c8101930 0x7f30c8199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.024+0000 7f30cfeac700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30c81022f0 0x7f30c81995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.024+0000 7f30cfeac700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c8107980 0x7f30c819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.024+0000 7f30cfeac700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30c8111ae0 con 0x7f30c81022f0 2024-04-25T11:43:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.024+0000 7f30cfeac700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f30c8111960 con 0x7f30c8107980 2024-04-25T11:43:36.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.024+0000 7f30cfeac700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f30c8111c60 con 0x7f30c8101930 2024-04-25T11:43:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.024+0000 7f30cd447700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30c81022f0 0x7f30c81995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.024+0000 7f30ce449700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c8107980 0x7f30c819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.024+0000 7f30cdc48700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30c8101930 0x7f30c8199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.024+0000 7f30cd447700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30c81022f0 0x7f30c81995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46644/0 (socket says 172.21.15.83:46644) 2024-04-25T11:43:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.024+0000 7f30cd447700 1 -- 172.21.15.83:0/3972162852 learned_addr learned my addr 172.21.15.83:0/3972162852 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.025+0000 7f30cdc48700 1 -- 172.21.15.83:0/3972162852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c8107980 msgr2=0x7f30c819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.025+0000 7f30cdc48700 1 --2- 172.21.15.83:0/3972162852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c8107980 0x7f30c819d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.025+0000 7f30cdc48700 1 -- 172.21.15.83:0/3972162852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30c81022f0 msgr2=0x7f30c81995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.025+0000 7f30cdc48700 1 --2- 172.21.15.83:0/3972162852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30c81022f0 0x7f30c81995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.025+0000 7f30cdc48700 1 -- 172.21.15.83:0/3972162852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c819e0a0 con 0x7f30c8101930 2024-04-25T11:43:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.025+0000 7f30ce449700 1 --2- 172.21.15.83:0/3972162852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c8107980 0x7f30c819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:43:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.025+0000 7f30cdc48700 1 --2- 172.21.15.83:0/3972162852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30c8101930 0x7f30c8199090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f30b800e9f0 tx=0x7f30b800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.025+0000 7f30beffd700 1 -- 172.21.15.83:0/3972162852 <== mon.1 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b800c9c0 con 0x7f30c8101930 2024-04-25T11:43:36.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.025+0000 7f30cfeac700 1 -- 172.21.15.83:0/3972162852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c819e390 con 0x7f30c8101930 2024-04-25T11:43:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.025+0000 7f30beffd700 1 -- 172.21.15.83:0/3972162852 <== mon.1 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30b800cb60 con 0x7f30c8101930 2024-04-25T11:43:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.025+0000 7f30cfeac700 1 -- 172.21.15.83:0/3972162852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c81a5c10 con 0x7f30c8101930 2024-04-25T11:43:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.025+0000 7f30beffd700 1 -- 172.21.15.83:0/3972162852 <== mon.1 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30b8018680 con 0x7f30c8101930 2024-04-25T11:43:36.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.026+0000 7f30cfeac700 1 -- 172.21.15.83:0/3972162852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c8069730 con 0x7f30c8101930 2024-04-25T11:43:36.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.027+0000 7f30beffd700 1 -- 172.21.15.83:0/3972162852 <== mon.1 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f30b8010430 con 0x7f30c8101930 2024-04-25T11:43:36.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.028+0000 7f30beffd700 1 --2- 172.21.15.83:0/3972162852 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f30b40a3d10 0x7f30b40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:36.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.028+0000 7f30cd447700 1 -- 172.21.15.83:0/3972162852 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f30b40a3d10 msgr2=0x7f30b40a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1157946519 2024-04-25T11:43:36.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.028+0000 7f30beffd700 1 -- 172.21.15.83:0/3972162852 <== mon.1 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f30b80d0560 con 0x7f30c8101930 2024-04-25T11:43:36.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.028+0000 7f30cd447700 1 --2- 172.21.15.83:0/3972162852 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f30b40a3d10 0x7f30b40a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:43:36.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.032+0000 7f30beffd700 1 -- 172.21.15.83:0/3972162852 <== mon.1 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30b809cef0 con 0x7f30c8101930 2024-04-25T11:43:36.019 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:35 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:35.844+0000 7efec1198200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:43:36.019 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:35 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:35.938+0000 7efec1198200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:43:36.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.229+0000 7f30cd447700 1 -- 172.21.15.83:0/3972162852 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f30b40a3d10 msgr2=0x7f30b40a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1157946519 2024-04-25T11:43:36.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.229+0000 7f30cd447700 1 --2- 172.21.15.83:0/3972162852 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f30b40a3d10 0x7f30b40a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:43:36.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.282+0000 7f30cfeac700 1 -- 172.21.15.83:0/3972162852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f30c802df80 con 0x7f30c8101930 2024-04-25T11:43:36.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.282+0000 7f30beffd700 1 -- 172.21.15.83:0/3972162852 <== mon.1 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f30b809c7f0 con 0x7f30c8101930 2024-04-25T11:43:36.269 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:43:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.284+0000 7f30cfeac700 1 -- 172.21.15.83:0/3972162852 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f30b40a3d10 msgr2=0x7f30b40a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:43:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.284+0000 7f30cfeac700 1 --2- 172.21.15.83:0/3972162852 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f30b40a3d10 0x7f30b40a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.284+0000 7f30cfeac700 1 -- 172.21.15.83:0/3972162852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30c8101930 msgr2=0x7f30c8199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.284+0000 7f30cfeac700 1 --2- 172.21.15.83:0/3972162852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30c8101930 0x7f30c8199090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f30b800e9f0 tx=0x7f30b800ed00 comp rx=0 tx=0).stop 2024-04-25T11:43:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.285+0000 7f30cfeac700 1 -- 172.21.15.83:0/3972162852 shutdown_connections 2024-04-25T11:43:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.285+0000 7f30cfeac700 1 --2- 172.21.15.83:0/3972162852 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f30b40a3d10 0x7f30b40a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.285+0000 7f30cfeac700 1 --2- 172.21.15.83:0/3972162852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30c8107980 0x7f30c819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.285+0000 7f30cfeac700 1 --2- 172.21.15.83:0/3972162852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30c8101930 0x7f30c8199090 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.285+0000 7f30cfeac700 1 --2- 172.21.15.83:0/3972162852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30c81022f0 0x7f30c81995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:36.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.285+0000 7f30cfeac700 1 -- 172.21.15.83:0/3972162852 >> 172.21.15.83:0/3972162852 conn(0x7f30c80788b0 msgr2=0x7f30c806b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:36.272 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:36 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:36.149+0000 7efec1198200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:43:36.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.285+0000 7f30cfeac700 1 -- 172.21.15.83:0/3972162852 shutdown_connections 2024-04-25T11:43:36.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36.285+0000 7f30cfeac700 1 -- 172.21.15.83:0/3972162852 wait complete. 2024-04-25T11:43:36.272 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:36 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:36.006+0000 7f029ad1a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:43:36.273 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:36 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:36.199+0000 7f029ad1a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:43:36.320 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:36 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:36.212+0000 7f202e97f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:43:36.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:36 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3972162852' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T11:43:36.567 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:36 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:36.351+0000 7efec1198200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:43:36.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:36 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3972162852' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T11:43:36.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:36 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3972162852' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T11:43:36.866 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:43:36.866 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:43:36.866 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-25T11:43:36.867 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:43:36.867 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:43:36.867 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:43:36.867 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:43:36.867 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:43:36.867 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:43:36.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:36.867 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-25T11:43:36.883 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:43:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:43:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:43:36.883 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:43:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-25T11:43:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:43:36.884 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:43:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-25T11:43:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-25T11:43:36.884 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:43:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:43:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-25T11:43:36.884 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:43:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:43:36.884 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:43:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:43:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T11:43:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42077sec preferred_lft 42077sec 2024-04-25T11:43:36.884 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ea47:558e:effa:6a90/64 scope link noprefixroute 2024-04-25T11:43:36.885 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:43:36.885 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T11:43:36.885 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:43:36.885 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T11:43:36.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add name ceph-brx type bridge 2024-04-25T11:43:36.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr flush dev ceph-brx 2024-04-25T11:43:36.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set ceph-brx up 2024-04-25T11:43:36.885 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T11:43:36.885 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T11:43:36.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:43:37.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:43:37.071 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:36 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:36.809+0000 7f202e97f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:43:37.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:37.073 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:43:37.093 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:36 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:36.968+0000 7efec1198200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:43:37.097 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-25T11:43:37.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:37.099 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-25T11:43:37.162 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-25T11:43:37.163 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:43:37.163 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:43:37.163 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:43:37.163 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:43:37.163 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:43:37.163 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T11:43:37.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:43:37.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:43:37.163 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:43:37.164 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T11:43:37.204 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:36 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:36.797+0000 7f029ad1a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:43:37.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:43:37.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:43:37.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:37.480 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T11:43:37.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:37.494 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-04-25T11:43:37.499 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:37 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:37.217+0000 7f029ad1a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:43:37.499 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:37 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:37.313+0000 7f029ad1a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:43:37.499 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:37 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:37.512+0000 7f029ad1a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:43:37.518 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:37 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:37.235+0000 7f202e97f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:43:37.519 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:37 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:37.332+0000 7f202e97f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:43:37.519 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:37 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:37.532+0000 7f202e97f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:43:37.549 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:43:37.549 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T11:43:37.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:43:37.549 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T11:43:37.549 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T11:43:37.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:43:37.646 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:37 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:37.426+0000 7efec1198200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:43:37.646 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:37 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:37.529+0000 7efec1198200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:43:37.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:43:37.668 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T11:43:37.668 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:43:37.668 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T11:43:37.668 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T11:43:37.668 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T11:43:37.668 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T11:43:37.668 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T11:43:37.668 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T11:43:37.668 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T11:43:37.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:43:37.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:43:37.795 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:43:37.795 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-25T11:43:37.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-25T11:43:37.795 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T11:43:37.795 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-25T11:43:37.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:43:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:43:37.871 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:43:37.871 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:43:37.886 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:43:37.886 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:43:37.886 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:43:37.904 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:37 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:37.748+0000 7efec1198200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:43:37.919 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T11:43:37.919 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T11:43:37.919 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/470219e6-04e0-4a93-9a4c-4bdb75bfa50d 2024-04-25T11:43:37.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:37.919 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/470219e6-04e0-4a93-9a4c-4bdb75bfa50d /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-25T11:43:38.008 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-25T11:43:38.008 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-25T11:43:38.008 INFO:teuthology.orchestra.run.smithi083.stdout:invalid new device string format 2024-04-25T11:43:38.132 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-25T11:43:38.132 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-25T11:43:38.133 INFO:teuthology.orchestra.run.smithi083.stdout:invalid new device string format 2024-04-25T11:43:38.133 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.94:3300,172.21.15.111:3300" 2024-04-25T11:43:38.133 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:3300,172.21.15.94:3300,172.21.15.111:3300:/volumes/qa/sv_1/470219e6-04e0-4a93-9a4c-4bdb75bfa50d 2024-04-25T11:43:38.133 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=7fcf54c6-02f7-11ef-bc93-c7b262605968" will pass to kernel 2024-04-25T11:43:38.134 INFO:teuthology.orchestra.run.smithi083.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:43:38.134 INFO:teuthology.orchestra.run.smithi083.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:43:38.134 INFO:teuthology.orchestra.run.smithi083.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:43:38.134 INFO:teuthology.orchestra.run.smithi083.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:43:38.134 INFO:teuthology.orchestra.run.smithi083.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:43:38.135 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:43:38.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:43:38.135 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:43:38.159 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:43:38.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:43:38.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:43:38.256 INFO:teuthology.orchestra.run.smithi083.stdout:client24880 2024-04-25T11:43:38.256 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:43:38.256 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:43:38.273 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:43:38.274 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:38 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:38.031+0000 7f029ad1a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:43:38.274 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:38 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:38.120+0000 7f029ad1a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:43:38.321 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:38.051+0000 7f202e97f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:43:38.321 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:38.143+0000 7f202e97f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:43:38.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:43:38.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:43:38.370 INFO:teuthology.orchestra.run.smithi083.stdout:client24880 2024-04-25T11:43:38.371 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:43:38.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:43:38.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:43:38.427 INFO:teuthology.orchestra.run.smithi083.stdout:client24880 2024-04-25T11:43:38.427 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968.client24880/status') 2024-04-25T11:43:38.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:43:38.486 INFO:teuthology.orchestra.run.smithi083.stdout:instance: client.24880 (3)172.21.15.83:0/1165495629 2024-04-25T11:43:38.486 INFO:teuthology.orchestra.run.smithi083.stdout:blocklisted: false 2024-04-25T11:43:38.487 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records in 2024-04-25T11:43:38.487 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records out 2024-04-25T11:43:38.487 INFO:teuthology.orchestra.run.smithi083.stderr:71 bytes copied, 2.3359e-05 s, 3.0 MB/s 2024-04-25T11:43:38.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:43:38.494 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T11:43:38.502 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T11:43:38.502 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T11:43:38.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-25T11:43:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.617+0000 7fdf786da640 1 -- 172.21.15.83:0/2881158600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf70103bf0 msgr2=0x7fdf7010a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.617+0000 7fdf786da640 1 --2- 172.21.15.83:0/2881158600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf70103bf0 0x7fdf7010a480 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009cd0 tx=0x7fdf6802a8e0 comp rx=0 tx=0).stop 2024-04-25T11:43:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf786da640 1 -- 172.21.15.83:0/2881158600 shutdown_connections 2024-04-25T11:43:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf786da640 1 --2- 172.21.15.83:0/2881158600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf70103bf0 0x7fdf7010a480 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf786da640 1 --2- 172.21.15.83:0/2881158600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf70103230 0x7fdf701036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf786da640 1 --2- 172.21.15.83:0/2881158600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf70100260 0x7fdf70100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf786da640 1 -- 172.21.15.83:0/2881158600 >> 172.21.15.83:0/2881158600 conn(0x7fdf700fbbb0 msgr2=0x7fdf700fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf786da640 1 -- 172.21.15.83:0/2881158600 shutdown_connections 2024-04-25T11:43:38.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf786da640 1 -- 172.21.15.83:0/2881158600 wait complete. 2024-04-25T11:43:38.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf786da640 1 Processor -- start 2024-04-25T11:43:38.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf786da640 1 -- start start 2024-04-25T11:43:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf786da640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf70100260 0x7fdf7019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf786da640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf70103230 0x7fdf7019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf786da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf70103bf0 0x7fdf701a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf786da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7010db00 con 0x7fdf70103bf0 2024-04-25T11:43:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf786da640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7010d980 con 0x7fdf70103230 2024-04-25T11:43:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf786da640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7010dc80 con 0x7fdf70100260 2024-04-25T11:43:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf76c77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf70103bf0 0x7fdf701a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf76c77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf70103bf0 0x7fdf701a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46670/0 (socket says 172.21.15.83:46670) 2024-04-25T11:43:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf76c77640 1 -- 172.21.15.83:0/2445860260 learned_addr learned my addr 172.21.15.83:0/2445860260 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.618+0000 7fdf76476640 1 --2- 172.21.15.83:0/2445860260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf70100260 0x7fdf7019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.619+0000 7fdf75c75640 1 --2- 172.21.15.83:0/2445860260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf70103230 0x7fdf7019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.619+0000 7fdf76c77640 1 -- 172.21.15.83:0/2445860260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf70100260 msgr2=0x7fdf7019a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.619+0000 7fdf76c77640 1 --2- 172.21.15.83:0/2445860260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf70100260 0x7fdf7019a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:38.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.619+0000 7fdf76c77640 1 -- 172.21.15.83:0/2445860260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf70103230 msgr2=0x7fdf7019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:38.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.619+0000 7fdf76c77640 1 --2- 172.21.15.83:0/2445860260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf70103230 0x7fdf7019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:38.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.619+0000 7fdf76c77640 1 -- 172.21.15.83:0/2445860260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf701a1f40 con 0x7fdf70103bf0 2024-04-25T11:43:38.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.619+0000 7fdf76c77640 1 --2- 172.21.15.83:0/2445860260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf70103bf0 0x7fdf701a18d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdf680318b0 tx=0x7fdf68003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:38.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.619+0000 7fdf677fe640 1 -- 172.21.15.83:0/2445860260 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68004040 con 0x7fdf70103bf0 2024-04-25T11:43:38.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.619+0000 7fdf786da640 1 -- 172.21.15.83:0/2445860260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf701a21d0 con 0x7fdf70103bf0 2024-04-25T11:43:38.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.619+0000 7fdf677fe640 1 -- 172.21.15.83:0/2445860260 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf68004690 con 0x7fdf70103bf0 2024-04-25T11:43:38.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.619+0000 7fdf677fe640 1 -- 172.21.15.83:0/2445860260 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6803b400 con 0x7fdf70103bf0 2024-04-25T11:43:38.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.619+0000 7fdf786da640 1 -- 172.21.15.83:0/2445860260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf701a2770 con 0x7fdf70103bf0 2024-04-25T11:43:38.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.620+0000 7fdf786da640 1 -- 172.21.15.83:0/2445860260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf701a2a90 con 0x7fdf70103bf0 2024-04-25T11:43:38.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.622+0000 7fdf677fe640 1 -- 172.21.15.83:0/2445860260 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fdf68030630 con 0x7fdf70103bf0 2024-04-25T11:43:38.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.622+0000 7fdf677fe640 1 --2- 172.21.15.83:0/2445860260 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdf5c0a3ca0 0x7fdf5c0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:38.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.622+0000 7fdf76476640 1 -- 172.21.15.83:0/2445860260 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdf5c0a3ca0 msgr2=0x7fdf5c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1157946519 2024-04-25T11:43:38.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.622+0000 7fdf677fe640 1 -- 172.21.15.83:0/2445860260 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7fdf6802e030 con 0x7fdf70103bf0 2024-04-25T11:43:38.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.622+0000 7fdf76476640 1 --2- 172.21.15.83:0/2445860260 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdf5c0a3ca0 0x7fdf5c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:43:38.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.627+0000 7fdf677fe640 1 -- 172.21.15.83:0/2445860260 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf680be510 con 0x7fdf70103bf0 2024-04-25T11:43:38.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:38 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:38.308+0000 7efec1198200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:43:38.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:38 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:38.408+0000 7efec1198200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:43:38.689 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:38 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:38.363+0000 7f029ad1a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:43:38.689 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:38 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:38.494+0000 7f029ad1a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:43:38.689 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:38 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:38.587+0000 7f029ad1a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:43:38.715 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:38.386+0000 7f202e97f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:43:38.715 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:38.518+0000 7f202e97f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:43:38.715 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:38.613+0000 7f202e97f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:43:38.715 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:38.729+0000 7f202e97f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:43:38.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.824+0000 7fdf76476640 1 -- 172.21.15.83:0/2445860260 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdf5c0a3ca0 msgr2=0x7fdf5c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1157946519 2024-04-25T11:43:38.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.824+0000 7fdf76476640 1 --2- 172.21.15.83:0/2445860260 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdf5c0a3ca0 0x7fdf5c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:43:38.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.849+0000 7fdf786da640 1 -- 172.21.15.83:0/2445860260 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fdf701a30a0 con 0x7fdf70103bf0 2024-04-25T11:43:38.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.901+0000 7fdf677fe640 1 -- 172.21.15.83:0/2445860260 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7fdf680bde10 con 0x7fdf70103bf0 2024-04-25T11:43:38.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.903+0000 7fdf786da640 1 -- 172.21.15.83:0/2445860260 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdf5c0a3ca0 msgr2=0x7fdf5c0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:43:38.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.903+0000 7fdf786da640 1 --2- 172.21.15.83:0/2445860260 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdf5c0a3ca0 0x7fdf5c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:38.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.903+0000 7fdf786da640 1 -- 172.21.15.83:0/2445860260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf70103bf0 msgr2=0x7fdf701a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:38.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.903+0000 7fdf786da640 1 --2- 172.21.15.83:0/2445860260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf70103bf0 0x7fdf701a18d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdf680318b0 tx=0x7fdf68003b80 comp rx=0 tx=0).stop 2024-04-25T11:43:38.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.903+0000 7fdf786da640 1 -- 172.21.15.83:0/2445860260 shutdown_connections 2024-04-25T11:43:38.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.903+0000 7fdf786da640 1 --2- 172.21.15.83:0/2445860260 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7fdf5c0a3ca0 0x7fdf5c0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:38.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.903+0000 7fdf786da640 1 --2- 172.21.15.83:0/2445860260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf70103bf0 0x7fdf701a18d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:38.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.903+0000 7fdf786da640 1 --2- 172.21.15.83:0/2445860260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf70103230 0x7fdf7019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:38.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.903+0000 7fdf786da640 1 --2- 172.21.15.83:0/2445860260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf70100260 0x7fdf7019a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:38.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.903+0000 7fdf786da640 1 -- 172.21.15.83:0/2445860260 >> 172.21.15.83:0/2445860260 conn(0x7fdf700fbbb0 msgr2=0x7fdf70101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:38.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.904+0000 7fdf786da640 1 -- 172.21.15.83:0/2445860260 shutdown_connections 2024-04-25T11:43:38.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:38.904+0000 7fdf786da640 1 -- 172.21.15.83:0/2445860260 wait complete. 2024-04-25T11:43:38.902 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T11:43:38.912 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T11:43:38.920 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T11:43:38.921 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T11:43:38.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T11:43:38.941 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:38 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2445860260' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T11:43:38.941 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:38 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:38.675+0000 7efec1198200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:43:38.941 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:38 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:38.814+0000 7efec1198200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:43:38.941 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:38 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:38.915+0000 7efec1198200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:43:38.988 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:38 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:38.703+0000 7f029ad1a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:43:38.988 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:38 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2445860260' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T11:43:39.013 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:38 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2445860260' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T11:43:39.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.037+0000 7f0d37ccf640 1 -- 172.21.15.83:0/515610650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d30100e60 msgr2=0x7f0d30101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:39.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.037+0000 7f0d37ccf640 1 --2- 172.21.15.83:0/515610650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d30100e60 0x7f0d30101280 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c009900 tx=0x7f0d2c02a410 comp rx=0 tx=0).stop 2024-04-25T11:43:39.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.037+0000 7f0d37ccf640 1 -- 172.21.15.83:0/515610650 shutdown_connections 2024-04-25T11:43:39.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.037+0000 7f0d37ccf640 1 --2- 172.21.15.83:0/515610650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d30102a00 0x7f0d30109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:39.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.037+0000 7f0d37ccf640 1 --2- 172.21.15.83:0/515610650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d30102040 0x7f0d301024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:39.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.037+0000 7f0d37ccf640 1 --2- 172.21.15.83:0/515610650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d30100e60 0x7f0d30101280 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:39.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.037+0000 7f0d37ccf640 1 -- 172.21.15.83:0/515610650 >> 172.21.15.83:0/515610650 conn(0x7f0d300fc420 msgr2=0x7f0d300fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:39.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d37ccf640 1 -- 172.21.15.83:0/515610650 shutdown_connections 2024-04-25T11:43:39.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d37ccf640 1 -- 172.21.15.83:0/515610650 wait complete. 2024-04-25T11:43:39.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d37ccf640 1 Processor -- start 2024-04-25T11:43:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d37ccf640 1 -- start start 2024-04-25T11:43:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d37ccf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d30100e60 0x7f0d3006a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d37ccf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d30102040 0x7f0d3019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d37ccf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d30102a00 0x7f0d3019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d37ccf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3010c9f0 con 0x7f0d30102a00 2024-04-25T11:43:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d37ccf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3010c870 con 0x7f0d30102040 2024-04-25T11:43:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d37ccf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3010cb70 con 0x7f0d30100e60 2024-04-25T11:43:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d3626c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d30102a00 0x7f0d3019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d35a6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d30100e60 0x7f0d3006a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d3526a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d30102040 0x7f0d3019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d35a6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d30100e60 0x7f0d3006a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.83:57796/0 (socket says 172.21.15.83:57796) 2024-04-25T11:43:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d35a6b640 1 -- 172.21.15.83:0/4116633182 learned_addr learned my addr 172.21.15.83:0/4116633182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.038+0000 7f0d3626c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d30102a00 0x7f0d3019f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46690/0 (socket says 172.21.15.83:46690) 2024-04-25T11:43:39.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.039+0000 7f0d3626c640 1 -- 172.21.15.83:0/4116633182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d30100e60 msgr2=0x7f0d3006a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.039+0000 7f0d3626c640 1 --2- 172.21.15.83:0/4116633182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d30100e60 0x7f0d3006a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.039+0000 7f0d3626c640 1 -- 172.21.15.83:0/4116633182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d30102040 msgr2=0x7f0d3019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.039+0000 7f0d3626c640 1 --2- 172.21.15.83:0/4116633182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d30102040 0x7f0d3019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.039+0000 7f0d3626c640 1 -- 172.21.15.83:0/4116633182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3019fdc0 con 0x7f0d30102a00 2024-04-25T11:43:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.039+0000 7f0d3526a640 1 --2- 172.21.15.83:0/4116633182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d30102040 0x7f0d3019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:43:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.039+0000 7f0d3626c640 1 --2- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d30102a00 0x7f0d3019f500 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009fd0 tx=0x7f0d2800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.039+0000 7f0d26ffd640 1 -- 172.21.15.83:0/4116633182 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2800ce20 con 0x7f0d30102a00 2024-04-25T11:43:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.039+0000 7f0d26ffd640 1 -- 172.21.15.83:0/4116633182 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d28005020 con 0x7f0d30102a00 2024-04-25T11:43:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.039+0000 7f0d26ffd640 1 -- 172.21.15.83:0/4116633182 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28010690 con 0x7f0d30102a00 2024-04-25T11:43:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.039+0000 7f0d37ccf640 1 -- 172.21.15.83:0/4116633182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d301a00b0 con 0x7f0d30102a00 2024-04-25T11:43:39.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.039+0000 7f0d37ccf640 1 -- 172.21.15.83:0/4116633182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d301a0600 con 0x7f0d30102a00 2024-04-25T11:43:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.040+0000 7f0d37ccf640 1 -- 172.21.15.83:0/4116633182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d301082d0 con 0x7f0d30102a00 2024-04-25T11:43:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.041+0000 7f0d26ffd640 1 -- 172.21.15.83:0/4116633182 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f0d280054d0 con 0x7f0d30102a00 2024-04-25T11:43:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.041+0000 7f0d26ffd640 1 --2- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0d1c0a3ca0 0x7f0d1c0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.042+0000 7f0d35a6b640 1 -- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0d1c0a3ca0 msgr2=0x7f0d1c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1157946519 2024-04-25T11:43:39.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.042+0000 7f0d26ffd640 1 -- 172.21.15.83:0/4116633182 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8211+0+0 (secure 0 0 0) 0x7f0d280d0aa0 con 0x7f0d30102a00 2024-04-25T11:43:39.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.042+0000 7f0d35a6b640 1 --2- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0d1c0a3ca0 0x7f0d1c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:43:39.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.046+0000 7f0d26ffd640 1 -- 172.21.15.83:0/4116633182 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d2809d490 con 0x7f0d30102a00 2024-04-25T11:43:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.243+0000 7f0d35a6b640 1 -- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0d1c0a3ca0 msgr2=0x7f0d1c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1157946519 2024-04-25T11:43:39.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.243+0000 7f0d35a6b640 1 --2- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0d1c0a3ca0 0x7f0d1c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:43:39.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.252+0000 7f0d37ccf640 1 -- 172.21.15.83:0/4116633182 --> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f0d30062390 con 0x7f0d1c0a3ca0 2024-04-25T11:43:39.271 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:39.002+0000 7f029ad1a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:43:39.321 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:39.026+0000 7f202e97f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:43:39.339 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:39.040+0000 7efec1198200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:43:39.339 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:39.353+0000 7efec1198200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:43:39.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.644+0000 7f0d35a6b640 1 -- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0d1c0a3ca0 msgr2=0x7f0d1c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1157946519 2024-04-25T11:43:39.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.644+0000 7f0d35a6b640 1 --2- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0d1c0a3ca0 0x7f0d1c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T11:43:39.893 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:39.598+0000 7f029ad1a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:43:39.893 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y[34423]: 2024-04-25T11:43:39.813+0000 7f029ad1a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:43:39.894 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:39.628+0000 7f202e97f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:43:39.894 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z[34627]: 2024-04-25T11:43:39.846+0000 7f202e97f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:43:39.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:39.918+0000 7f0d26ffd640 1 -- 172.21.15.83:0/4116633182 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f0d28099400 con 0x7f0d30102a00 2024-04-25T11:43:40.155 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:39.974+0000 7efec1198200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:43:40.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2445860260' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T11:43:40.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-mon[29330]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T11:43:40.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-mon[29330]: Standby manager daemon y restarted 2024-04-25T11:43:40.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-mon[29330]: Standby manager daemon y started 2024-04-25T11:43:40.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.94:0/171427778' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T11:43:40.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.94:0/171427778' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:43:40.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.94:0/171427778' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T11:43:40.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.94:0/171427778' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:43:40.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-mon[29330]: Standby manager daemon z restarted 2024-04-25T11:43:40.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-mon[29330]: Standby manager daemon z started 2024-04-25T11:43:40.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.111:0/2569184661' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T11:43:40.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.111:0/2569184661' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:43:40.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.111:0/2569184661' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T11:43:40.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:39 smithi083 ceph-mon[29330]: from='mgr.? 172.21.15.111:0/2569184661' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:43:40.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:40.210+0000 7f0d26ffd640 1 -- 172.21.15.83:0/4116633182 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f0d280d2ae0 con 0x7f0d30102a00 2024-04-25T11:43:40.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:40.210+0000 7f0d26ffd640 1 -- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0d1c0a3ca0 msgr2=0x7f0d1c0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:43:40.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:40.210+0000 7f0d26ffd640 1 --2- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:6800/1157946519,v1:172.21.15.83:6801/1157946519] conn(0x7f0d1c0a3ca0 0x7f0d1c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:40.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2445860260' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T11:43:40.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-mon[33032]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T11:43:40.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-mon[33032]: Standby manager daemon y restarted 2024-04-25T11:43:40.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-mon[33032]: Standby manager daemon y started 2024-04-25T11:43:40.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.94:0/171427778' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T11:43:40.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.94:0/171427778' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:43:40.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.94:0/171427778' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T11:43:40.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.94:0/171427778' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:43:40.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-mon[33032]: Standby manager daemon z restarted 2024-04-25T11:43:40.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-mon[33032]: Standby manager daemon z started 2024-04-25T11:43:40.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.111:0/2569184661' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T11:43:40.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.111:0/2569184661' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:43:40.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.111:0/2569184661' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T11:43:40.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:39 smithi094 ceph-mon[33032]: from='mgr.? 172.21.15.111:0/2569184661' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:43:40.312 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2445860260' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T11:43:40.313 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-mon[32784]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-25T11:43:40.313 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-mon[32784]: Standby manager daemon y restarted 2024-04-25T11:43:40.313 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-mon[32784]: Standby manager daemon y started 2024-04-25T11:43:40.313 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.94:0/171427778' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T11:43:40.313 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.94:0/171427778' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:43:40.313 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.94:0/171427778' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T11:43:40.313 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.94:0/171427778' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:43:40.313 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-mon[32784]: Standby manager daemon z restarted 2024-04-25T11:43:40.313 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-mon[32784]: Standby manager daemon z started 2024-04-25T11:43:40.313 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.111:0/2569184661' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T11:43:40.313 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.111:0/2569184661' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T11:43:40.313 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.111:0/2569184661' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T11:43:40.313 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:39 smithi111 ceph-mon[32784]: from='mgr.? 172.21.15.111:0/2569184661' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T11:43:40.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:40.197+0000 7efec1198200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:43:40.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:40.433+0000 7efe22b10700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:43:40.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:40.433+0000 7efe22b10700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:43:40.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:40.433+0000 7efe22b10700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:43:40.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:40.433+0000 7efe22b10700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:43:40.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:40.433+0000 7efe22b10700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:43:40.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:40.434+0000 7efe2030b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:43:40.655 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:40.434+0000 7efe2030b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:43:40.656 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:40.434+0000 7efe2030b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:43:40.656 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:40.434+0000 7efe2030b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:43:40.656 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:43:40.434+0000 7efe2030b700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:43:41.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-04-25T11:43:41.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: Active manager daemon x restarted 2024-04-25T11:43:41.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: Activating manager daemon x 2024-04-25T11:43:41.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T11:43:41.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: mgrmap e21: x(active, starting, since 0.0129622s), standbys: z, y 2024-04-25T11:43:41.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:43:41.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:43:41.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:43:41.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T11:43:41.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T11:43:41.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T11:43:41.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T11:43:41.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T11:43:41.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T11:43:41.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T11:43:41.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T11:43:41.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T11:43:41.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T11:43:41.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:43:41.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:43:41.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T11:43:41.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T11:43:41.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T11:43:41.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:43:41.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:43:41.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:43:41.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:43:41.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:43:41.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:43:41.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:43:41.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:43:41.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:43:41.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:43:41.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:43:41.160 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:43:41.160 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T11:43:41.160 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T11:43:41.160 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T11:43:41.160 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: Manager daemon x is now available 2024-04-25T11:43:41.160 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:43:41.160 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T11:43:41.160 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:40 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T11:43:41.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.218+0000 7f0d26ffd640 1 -- 172.21.15.83:0/4116633182 <== mon.0 v2:172.21.15.83:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f0d2805f300 con 0x7f0d30102a00 2024-04-25T11:43:41.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.219+0000 7f0d26ffd640 1 --2- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f0d1c035890 0x7f0d1c035c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:41.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.219+0000 7f0d26ffd640 1 -- 172.21.15.83:0/4116633182 --> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f0d30062390 con 0x7f0d1c035890 2024-04-25T11:43:41.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.220+0000 7f0d35a6b640 1 --2- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f0d1c035890 0x7f0d1c035c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:41.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.223+0000 7f0d35a6b640 1 --2- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f0d1c035890 0x7f0d1c035c70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c000c00 tx=0x7f0d2c0314a0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:41.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-04-25T11:43:41.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: Active manager daemon x restarted 2024-04-25T11:43:41.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: Activating manager daemon x 2024-04-25T11:43:41.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T11:43:41.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: mgrmap e21: x(active, starting, since 0.0129622s), standbys: z, y 2024-04-25T11:43:41.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:43:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:43:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:43:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T11:43:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T11:43:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T11:43:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T11:43:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T11:43:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T11:43:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T11:43:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T11:43:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T11:43:41.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T11:43:41.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:43:41.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:43:41.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T11:43:41.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T11:43:41.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T11:43:41.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:43:41.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:43:41.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:43:41.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:43:41.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:43:41.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:43:41.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:43:41.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:43:41.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:43:41.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:43:41.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:43:41.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:43:41.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T11:43:41.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T11:43:41.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T11:43:41.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: Manager daemon x is now available 2024-04-25T11:43:41.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:43:41.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T11:43:41.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:40 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T11:43:41.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.286+0000 7f0d26ffd640 1 -- 172.21.15.83:0/4116633182 <== mgr.15360 v2:172.21.15.83:6800/4112952368 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0d30062390 con 0x7f0d1c035890 2024-04-25T11:43:41.276 INFO:teuthology.orchestra.run.smithi083.stdout:Schedule set for path / 2024-04-25T11:43:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.288+0000 7f0d37ccf640 1 -- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f0d1c035890 msgr2=0x7f0d1c035c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.288+0000 7f0d37ccf640 1 --2- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f0d1c035890 0x7f0d1c035c70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c000c00 tx=0x7f0d2c0314a0 comp rx=0 tx=0).stop 2024-04-25T11:43:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.288+0000 7f0d37ccf640 1 -- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d30102a00 msgr2=0x7f0d3019f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.288+0000 7f0d37ccf640 1 --2- 172.21.15.83:0/4116633182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d30102a00 0x7f0d3019f500 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009fd0 tx=0x7f0d2800ef10 comp rx=0 tx=0).stop 2024-04-25T11:43:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.288+0000 7f0d3626c640 1 -- 172.21.15.83:0/4116633182 reap_dead start 2024-04-25T11:43:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.288+0000 7f0d37ccf640 1 -- 172.21.15.83:0/4116633182 shutdown_connections 2024-04-25T11:43:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.288+0000 7f0d37ccf640 1 -- 172.21.15.83:0/4116633182 >> 172.21.15.83:0/4116633182 conn(0x7f0d300fc420 msgr2=0x7f0d301054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.288+0000 7f0d37ccf640 1 -- 172.21.15.83:0/4116633182 shutdown_connections 2024-04-25T11:43:41.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.288+0000 7f0d37ccf640 1 -- 172.21.15.83:0/4116633182 wait complete. 2024-04-25T11:43:41.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T11:43:41.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-04-25T11:43:41.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: Active manager daemon x restarted 2024-04-25T11:43:41.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: Activating manager daemon x 2024-04-25T11:43:41.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T11:43:41.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: mgrmap e21: x(active, starting, since 0.0129622s), standbys: z, y 2024-04-25T11:43:41.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T11:43:41.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T11:43:41.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T11:43:41.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T11:43:41.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T11:43:41.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T11:43:41.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T11:43:41.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T11:43:41.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T11:43:41.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T11:43:41.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T11:43:41.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T11:43:41.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T11:43:41.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:43:41.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:43:41.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T11:43:41.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T11:43:41.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T11:43:41.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T11:43:41.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T11:43:41.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T11:43:41.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T11:43:41.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T11:43:41.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T11:43:41.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T11:43:41.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T11:43:41.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T11:43:41.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T11:43:41.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T11:43:41.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T11:43:41.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T11:43:41.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T11:43:41.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T11:43:41.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: Manager daemon x is now available 2024-04-25T11:43:41.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:43:41.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T11:43:41.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:40 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T11:43:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.444+0000 7f81b3c31640 1 -- 172.21.15.83:0/3612937967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac102a50 msgr2=0x7f81ac104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.444+0000 7f81b3c31640 1 --2- 172.21.15.83:0/3612937967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac102a50 0x7f81ac104e40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009900 tx=0x7f81a802a410 comp rx=0 tx=0).stop 2024-04-25T11:43:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.444+0000 7f81b3c31640 1 -- 172.21.15.83:0/3612937967 shutdown_connections 2024-04-25T11:43:41.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.444+0000 7f81b3c31640 1 --2- 172.21.15.83:0/3612937967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81ac107cb0 0x7f81ac10a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.444+0000 7f81b3c31640 1 --2- 172.21.15.83:0/3612937967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81ac105380 0x7f81ac107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.444+0000 7f81b3c31640 1 --2- 172.21.15.83:0/3612937967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac102a50 0x7f81ac104e40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.444+0000 7f81b3c31640 1 -- 172.21.15.83:0/3612937967 >> 172.21.15.83:0/3612937967 conn(0x7f81ac0fc440 msgr2=0x7f81ac0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.444+0000 7f81b3c31640 1 -- 172.21.15.83:0/3612937967 shutdown_connections 2024-04-25T11:43:41.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.444+0000 7f81b3c31640 1 -- 172.21.15.83:0/3612937967 wait complete. 2024-04-25T11:43:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b3c31640 1 Processor -- start 2024-04-25T11:43:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b3c31640 1 -- start start 2024-04-25T11:43:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b3c31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81ac102a50 0x7f81ac198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b3c31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac105380 0x7f81ac198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b3c31640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81ac107cb0 0x7f81ac19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b3c31640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac10db60 con 0x7f81ac105380 2024-04-25T11:43:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b3c31640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac10d9e0 con 0x7f81ac102a50 2024-04-25T11:43:41.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b3c31640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac10dce0 con 0x7f81ac107cb0 2024-04-25T11:43:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b11cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac105380 0x7f81ac198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b11cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac105380 0x7f81ac198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46838/0 (socket says 172.21.15.83:46838) 2024-04-25T11:43:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b21ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81ac107cb0 0x7f81ac19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b11cc640 1 -- 172.21.15.83:0/674406042 learned_addr learned my addr 172.21.15.83:0/674406042 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b19cd640 1 --2- 172.21.15.83:0/674406042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81ac102a50 0x7f81ac198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b11cc640 1 -- 172.21.15.83:0/674406042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81ac107cb0 msgr2=0x7f81ac19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b11cc640 1 --2- 172.21.15.83:0/674406042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81ac107cb0 0x7f81ac19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b11cc640 1 -- 172.21.15.83:0/674406042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81ac102a50 msgr2=0x7f81ac198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b11cc640 1 --2- 172.21.15.83:0/674406042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81ac102a50 0x7f81ac198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b11cc640 1 -- 172.21.15.83:0/674406042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ac19fd70 con 0x7f81ac105380 2024-04-25T11:43:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b21ce640 1 --2- 172.21.15.83:0/674406042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81ac107cb0 0x7f81ac19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:43:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b19cd640 1 --2- 172.21.15.83:0/674406042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81ac102a50 0x7f81ac198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:43:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81b11cc640 1 --2- 172.21.15.83:0/674406042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac105380 0x7f81ac198690 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f819c00e9c0 tx=0x7f819c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:41.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.445+0000 7f81a2ffd640 1 -- 172.21.15.83:0/674406042 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c00cd80 con 0x7f81ac105380 2024-04-25T11:43:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.446+0000 7f81a2ffd640 1 -- 172.21.15.83:0/674406042 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f819c0054d0 con 0x7f81ac105380 2024-04-25T11:43:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.446+0000 7f81b3c31640 1 -- 172.21.15.83:0/674406042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ac1a0060 con 0x7f81ac105380 2024-04-25T11:43:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.446+0000 7f81a2ffd640 1 -- 172.21.15.83:0/674406042 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c010430 con 0x7f81ac105380 2024-04-25T11:43:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.446+0000 7f81b3c31640 1 -- 172.21.15.83:0/674406042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ac1a05b0 con 0x7f81ac105380 2024-04-25T11:43:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.447+0000 7f81b3c31640 1 -- 172.21.15.83:0/674406042 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81ac0fdf30 con 0x7f81ac105380 2024-04-25T11:43:41.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.448+0000 7f81a2ffd640 1 -- 172.21.15.83:0/674406042 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f819c004fd0 con 0x7f81ac105380 2024-04-25T11:43:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.448+0000 7f81a2ffd640 1 --2- 172.21.15.83:0/674406042 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f81980a3a60 0x7f81980a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.448+0000 7f81b19cd640 1 --2- 172.21.15.83:0/674406042 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f81980a3a60 0x7f81980a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.448+0000 7f81a2ffd640 1 -- 172.21.15.83:0/674406042 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f819c0d04e0 con 0x7f81ac105380 2024-04-25T11:43:41.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.449+0000 7f81b19cd640 1 --2- 172.21.15.83:0/674406042 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f81980a3a60 0x7f81980a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f81a8031500 tx=0x7f81a8005a20 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:41.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.453+0000 7f81a2ffd640 1 -- 172.21.15.83:0/674406042 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f819c09d0c0 con 0x7f81ac105380 2024-04-25T11:43:41.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.662+0000 7f81b3c31640 1 -- 172.21.15.83:0/674406042 --> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f81ac062430 con 0x7f81980a3a60 2024-04-25T11:43:41.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.719+0000 7f81a2ffd640 1 -- 172.21.15.83:0/674406042 <== mgr.15360 v2:172.21.15.83:6800/4112952368 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f81ac062430 con 0x7f81980a3a60 2024-04-25T11:43:41.706 INFO:teuthology.orchestra.run.smithi083.stdout:Retention added to path / 2024-04-25T11:43:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.721+0000 7f81b3c31640 1 -- 172.21.15.83:0/674406042 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f81980a3a60 msgr2=0x7f81980a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.721+0000 7f81b3c31640 1 --2- 172.21.15.83:0/674406042 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f81980a3a60 0x7f81980a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f81a8031500 tx=0x7f81a8005a20 comp rx=0 tx=0).stop 2024-04-25T11:43:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.721+0000 7f81b3c31640 1 -- 172.21.15.83:0/674406042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac105380 msgr2=0x7f81ac198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.721+0000 7f81b3c31640 1 --2- 172.21.15.83:0/674406042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac105380 0x7f81ac198690 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f819c00e9c0 tx=0x7f819c00ee80 comp rx=0 tx=0).stop 2024-04-25T11:43:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.721+0000 7f81b3c31640 1 -- 172.21.15.83:0/674406042 shutdown_connections 2024-04-25T11:43:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.721+0000 7f81b3c31640 1 --2- 172.21.15.83:0/674406042 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f81980a3a60 0x7f81980a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.721+0000 7f81b3c31640 1 --2- 172.21.15.83:0/674406042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81ac105380 0x7f81ac198690 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.721+0000 7f81b3c31640 1 --2- 172.21.15.83:0/674406042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81ac107cb0 0x7f81ac19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.721+0000 7f81b3c31640 1 --2- 172.21.15.83:0/674406042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f81ac102a50 0x7f81ac198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.721+0000 7f81b3c31640 1 -- 172.21.15.83:0/674406042 >> 172.21.15.83:0/674406042 conn(0x7f81ac0fc440 msgr2=0x7f81ac103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.721+0000 7f81b3c31640 1 -- 172.21.15.83:0/674406042 shutdown_connections 2024-04-25T11:43:41.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.721+0000 7f81b3c31640 1 -- 172.21.15.83:0/674406042 wait complete. 2024-04-25T11:43:41.719 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T11:43:41.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.831+0000 7f834915d640 1 -- 172.21.15.83:0/418348876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8344102040 msgr2=0x7f83441024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:41.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.831+0000 7f834915d640 1 --2- 172.21.15.83:0/418348876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8344102040 0x7f83441024c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8338009900 tx=0x7f833802a410 comp rx=0 tx=0).stop 2024-04-25T11:43:41.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.831+0000 7f834915d640 1 -- 172.21.15.83:0/418348876 shutdown_connections 2024-04-25T11:43:41.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.831+0000 7f834915d640 1 --2- 172.21.15.83:0/418348876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8344102040 0x7f83441024c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:41.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.831+0000 7f834915d640 1 --2- 172.21.15.83:0/418348876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8344102a00 0x7f8344109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:41.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.831+0000 7f834915d640 1 --2- 172.21.15.83:0/418348876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8344100e60 0x7f8344101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:41.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.831+0000 7f834915d640 1 -- 172.21.15.83:0/418348876 >> 172.21.15.83:0/418348876 conn(0x7f83440fc420 msgr2=0x7f83440fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:41.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.831+0000 7f834915d640 1 -- 172.21.15.83:0/418348876 shutdown_connections 2024-04-25T11:43:41.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.831+0000 7f834915d640 1 -- 172.21.15.83:0/418348876 wait complete. 2024-04-25T11:43:41.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.832+0000 7f834915d640 1 Processor -- start 2024-04-25T11:43:41.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.832+0000 7f834915d640 1 -- start start 2024-04-25T11:43:41.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.832+0000 7f834915d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8344100e60 0x7f83441980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:41.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.832+0000 7f834915d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8344102040 0x7f8344198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:41.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.832+0000 7f834915d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8344102a00 0x7f834419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:41.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.832+0000 7f834915d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f834410db30 con 0x7f8344102040 2024-04-25T11:43:41.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.832+0000 7f834915d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f834410d9b0 con 0x7f8344102a00 2024-04-25T11:43:41.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.832+0000 7f834915d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f834410dcb0 con 0x7f8344100e60 2024-04-25T11:43:41.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.832+0000 7f834259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8344102040 0x7f8344198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:41.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.832+0000 7f834259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8344102040 0x7f8344198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46848/0 (socket says 172.21.15.83:46848) 2024-04-25T11:43:41.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.832+0000 7f834259c640 1 -- 172.21.15.83:0/3567905566 learned_addr learned my addr 172.21.15.83:0/3567905566 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:41.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.832+0000 7f8342d9d640 1 --2- 172.21.15.83:0/3567905566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8344100e60 0x7f83441980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:41.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.833+0000 7f834359e640 1 --2- 172.21.15.83:0/3567905566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8344102a00 0x7f834419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:41.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.833+0000 7f834259c640 1 -- 172.21.15.83:0/3567905566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8344100e60 msgr2=0x7f83441980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:41.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.833+0000 7f834259c640 1 --2- 172.21.15.83:0/3567905566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8344100e60 0x7f83441980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:41.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.833+0000 7f834259c640 1 -- 172.21.15.83:0/3567905566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8344102a00 msgr2=0x7f834419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:41.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.833+0000 7f834259c640 1 --2- 172.21.15.83:0/3567905566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8344102a00 0x7f834419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:41.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.833+0000 7f834259c640 1 -- 172.21.15.83:0/3567905566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834419fd10 con 0x7f8344102040 2024-04-25T11:43:41.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.833+0000 7f834259c640 1 --2- 172.21.15.83:0/3567905566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8344102040 0x7f8344198630 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8338031500 tx=0x7f83380043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:41.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.833+0000 7f832bfff640 1 -- 172.21.15.83:0/3567905566 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833803f030 con 0x7f8344102040 2024-04-25T11:43:41.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.833+0000 7f834915d640 1 -- 172.21.15.83:0/3567905566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f834419ffa0 con 0x7f8344102040 2024-04-25T11:43:41.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.833+0000 7f832bfff640 1 -- 172.21.15.83:0/3567905566 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8338003800 con 0x7f8344102040 2024-04-25T11:43:41.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.833+0000 7f832bfff640 1 -- 172.21.15.83:0/3567905566 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338035070 con 0x7f8344102040 2024-04-25T11:43:41.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.833+0000 7f834915d640 1 -- 172.21.15.83:0/3567905566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83441a0410 con 0x7f8344102040 2024-04-25T11:43:41.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.834+0000 7f834915d640 1 -- 172.21.15.83:0/3567905566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8344108200 con 0x7f8344102040 2024-04-25T11:43:41.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.835+0000 7f832bfff640 1 -- 172.21.15.83:0/3567905566 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f83380039b0 con 0x7f8344102040 2024-04-25T11:43:41.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.836+0000 7f832bfff640 1 --2- 172.21.15.83:0/3567905566 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f832c0a3a60 0x7f832c0a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:41.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.836+0000 7f8342d9d640 1 --2- 172.21.15.83:0/3567905566 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f832c0a3a60 0x7f832c0a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:41.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.836+0000 7f832bfff640 1 -- 172.21.15.83:0/3567905566 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f83380f0a50 con 0x7f8344102040 2024-04-25T11:43:41.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.836+0000 7f8342d9d640 1 --2- 172.21.15.83:0/3567905566 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f832c0a3a60 0x7f832c0a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8344101bb0 tx=0x7f8334007600 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:41.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:41.840+0000 7f832bfff640 1 -- 172.21.15.83:0/3567905566 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f83380bcfe0 con 0x7f8344102040 2024-04-25T11:43:42.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.058+0000 7f834915d640 1 -- 172.21.15.83:0/3567905566 --> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f8344062390 con 0x7f832c0a3a60 2024-04-25T11:43:42.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.066+0000 7f832bfff640 1 -- 172.21.15.83:0/3567905566 <== mgr.15360 v2:172.21.15.83:6800/4112952368 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f8344062390 con 0x7f832c0a3a60 2024-04-25T11:43:42.053 INFO:teuthology.orchestra.run.smithi083.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T11:43:41", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T11:43:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.068+0000 7f834915d640 1 -- 172.21.15.83:0/3567905566 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f832c0a3a60 msgr2=0x7f832c0a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.068+0000 7f834915d640 1 --2- 172.21.15.83:0/3567905566 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f832c0a3a60 0x7f832c0a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8344101bb0 tx=0x7f8334007600 comp rx=0 tx=0).stop 2024-04-25T11:43:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.068+0000 7f834915d640 1 -- 172.21.15.83:0/3567905566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8344102040 msgr2=0x7f8344198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.068+0000 7f834915d640 1 --2- 172.21.15.83:0/3567905566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8344102040 0x7f8344198630 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8338031500 tx=0x7f83380043c0 comp rx=0 tx=0).stop 2024-04-25T11:43:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.068+0000 7f834915d640 1 -- 172.21.15.83:0/3567905566 shutdown_connections 2024-04-25T11:43:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.068+0000 7f834915d640 1 --2- 172.21.15.83:0/3567905566 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f832c0a3a60 0x7f832c0a5f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.068+0000 7f834915d640 1 --2- 172.21.15.83:0/3567905566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8344102040 0x7f8344198630 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.068+0000 7f834915d640 1 --2- 172.21.15.83:0/3567905566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8344102a00 0x7f834419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.068+0000 7f834915d640 1 --2- 172.21.15.83:0/3567905566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8344100e60 0x7f83441980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.068+0000 7f834915d640 1 -- 172.21.15.83:0/3567905566 >> 172.21.15.83:0/3567905566 conn(0x7f83440fc420 msgr2=0x7f83441053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.068+0000 7f834915d640 1 -- 172.21.15.83:0/3567905566 shutdown_connections 2024-04-25T11:43:42.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.068+0000 7f834915d640 1 -- 172.21.15.83:0/3567905566 wait complete. 2024-04-25T11:43:42.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T11:43:42.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:41 smithi083 ceph-mon[29330]: [25/Apr/2024:11:43:40] ENGINE Bus STARTING 2024-04-25T11:43:42.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:41 smithi083 ceph-mon[29330]: [25/Apr/2024:11:43:40] ENGINE Serving on https://172.21.15.83:7150 2024-04-25T11:43:42.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:41 smithi083 ceph-mon[29330]: [25/Apr/2024:11:43:40] ENGINE Serving on http://172.21.15.83:8765 2024-04-25T11:43:42.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:41 smithi083 ceph-mon[29330]: [25/Apr/2024:11:43:40] ENGINE Bus STARTED 2024-04-25T11:43:42.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:41 smithi083 ceph-mon[29330]: mgrmap e22: x(active, since 1.02047s), standbys: z, y 2024-04-25T11:43:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.189+0000 7f2fb4b43640 1 -- 172.21.15.83:0/3431205236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb0100030 msgr2=0x7f2fb01004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.189+0000 7f2fb4b43640 1 --2- 172.21.15.83:0/3431205236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb0100030 0x7f2fb01004b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009900 tx=0x7f2f9802a410 comp rx=0 tx=0).stop 2024-04-25T11:43:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.189+0000 7f2fb4b43640 1 -- 172.21.15.83:0/3431205236 shutdown_connections 2024-04-25T11:43:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.189+0000 7f2fb4b43640 1 --2- 172.21.15.83:0/3431205236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb0100030 0x7f2fb01004b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.189+0000 7f2fb4b43640 1 --2- 172.21.15.83:0/3431205236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fb01009f0 0x7f2fb010a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.189+0000 7f2fb4b43640 1 --2- 172.21.15.83:0/3431205236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fb01035c0 0x7f2fb01039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.189+0000 7f2fb4b43640 1 -- 172.21.15.83:0/3431205236 >> 172.21.15.83:0/3431205236 conn(0x7f2fb00fbbb0 msgr2=0x7f2fb00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.189+0000 7f2fb4b43640 1 -- 172.21.15.83:0/3431205236 shutdown_connections 2024-04-25T11:43:42.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.190+0000 7f2fb4b43640 1 -- 172.21.15.83:0/3431205236 wait complete. 2024-04-25T11:43:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.190+0000 7f2fb4b43640 1 Processor -- start 2024-04-25T11:43:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.190+0000 7f2fb4b43640 1 -- start start 2024-04-25T11:43:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fb4b43640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb0100030 0x7f2fb019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fb4b43640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fb01009f0 0x7f2fb019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fb4b43640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fb01035c0 0x7f2fb01a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fb4b43640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb010db00 con 0x7f2fb0100030 2024-04-25T11:43:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fb4b43640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb010d980 con 0x7f2fb01035c0 2024-04-25T11:43:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fb4b43640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb010dc80 con 0x7f2fb01009f0 2024-04-25T11:43:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fae59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb0100030 0x7f2fb019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fa7fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fb01009f0 0x7f2fb019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:42.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fae59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb0100030 0x7f2fb019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46864/0 (socket says 172.21.15.83:46864) 2024-04-25T11:43:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fae59c640 1 -- 172.21.15.83:0/3710940726 learned_addr learned my addr 172.21.15.83:0/3710940726 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:43:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2faed9d640 1 --2- 172.21.15.83:0/3710940726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fb01035c0 0x7f2fb01a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fae59c640 1 -- 172.21.15.83:0/3710940726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fb01009f0 msgr2=0x7f2fb019a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fae59c640 1 --2- 172.21.15.83:0/3710940726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fb01009f0 0x7f2fb019a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fae59c640 1 -- 172.21.15.83:0/3710940726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fb01035c0 msgr2=0x7f2fb01a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fae59c640 1 --2- 172.21.15.83:0/3710940726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fb01035c0 0x7f2fb01a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fae59c640 1 -- 172.21.15.83:0/3710940726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb01a1f70 con 0x7f2fb0100030 2024-04-25T11:43:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2faed9d640 1 --2- 172.21.15.83:0/3710940726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fb01035c0 0x7f2fb01a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:43:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fa7fff640 1 --2- 172.21.15.83:0/3710940726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fb01009f0 0x7f2fb019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:43:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fae59c640 1 --2- 172.21.15.83:0/3710940726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb0100030 0x7f2fb019a350 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000e9c0 tx=0x7f2fa000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fa77fe640 1 -- 172.21.15.83:0/3710940726 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa000cd80 con 0x7f2fb0100030 2024-04-25T11:43:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fb4b43640 1 -- 172.21.15.83:0/3710940726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb01a2260 con 0x7f2fb0100030 2024-04-25T11:43:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fa77fe640 1 -- 172.21.15.83:0/3710940726 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fa00035f0 con 0x7f2fb0100030 2024-04-25T11:43:42.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.191+0000 7f2fa77fe640 1 -- 172.21.15.83:0/3710940726 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0010690 con 0x7f2fb0100030 2024-04-25T11:43:42.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.192+0000 7f2fb4b43640 1 -- 172.21.15.83:0/3710940726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb01a27d0 con 0x7f2fb0100030 2024-04-25T11:43:42.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.193+0000 7f2fb4b43640 1 -- 172.21.15.83:0/3710940726 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb004fce0 con 0x7f2fb0100030 2024-04-25T11:43:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.193+0000 7f2fa77fe640 1 -- 172.21.15.83:0/3710940726 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f2fa0004020 con 0x7f2fb0100030 2024-04-25T11:43:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.194+0000 7f2fa77fe640 1 --2- 172.21.15.83:0/3710940726 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f2f9c0a3a60 0x7f2f9c0a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:43:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.194+0000 7f2fa7fff640 1 --2- 172.21.15.83:0/3710940726 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f2f9c0a3a60 0x7f2f9c0a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:43:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.194+0000 7f2fa77fe640 1 -- 172.21.15.83:0/3710940726 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8469+0+0 (secure 0 0 0) 0x7f2fa00d0420 con 0x7f2fb0100030 2024-04-25T11:43:42.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.195+0000 7f2fa7fff640 1 --2- 172.21.15.83:0/3710940726 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f2f9c0a3a60 0x7f2f9c0a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2f98000c00 tx=0x7f2f980314a0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:43:42.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.199+0000 7f2fa77fe640 1 -- 172.21.15.83:0/3710940726 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2fa009d000 con 0x7f2fb0100030 2024-04-25T11:43:42.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:41 smithi094 ceph-mon[33032]: [25/Apr/2024:11:43:40] ENGINE Bus STARTING 2024-04-25T11:43:42.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:41 smithi094 ceph-mon[33032]: [25/Apr/2024:11:43:40] ENGINE Serving on https://172.21.15.83:7150 2024-04-25T11:43:42.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:41 smithi094 ceph-mon[33032]: [25/Apr/2024:11:43:40] ENGINE Serving on http://172.21.15.83:8765 2024-04-25T11:43:42.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:41 smithi094 ceph-mon[33032]: [25/Apr/2024:11:43:40] ENGINE Bus STARTED 2024-04-25T11:43:42.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:41 smithi094 ceph-mon[33032]: mgrmap e22: x(active, since 1.02047s), standbys: z, y 2024-04-25T11:43:42.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:41 smithi111 ceph-mon[32784]: [25/Apr/2024:11:43:40] ENGINE Bus STARTING 2024-04-25T11:43:42.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:41 smithi111 ceph-mon[32784]: [25/Apr/2024:11:43:40] ENGINE Serving on https://172.21.15.83:7150 2024-04-25T11:43:42.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:41 smithi111 ceph-mon[32784]: [25/Apr/2024:11:43:40] ENGINE Serving on http://172.21.15.83:8765 2024-04-25T11:43:42.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:41 smithi111 ceph-mon[32784]: [25/Apr/2024:11:43:40] ENGINE Bus STARTED 2024-04-25T11:43:42.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:41 smithi111 ceph-mon[32784]: mgrmap e22: x(active, since 1.02047s), standbys: z, y 2024-04-25T11:43:42.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.429+0000 7f2fb4b43640 1 -- 172.21.15.83:0/3710940726 --> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2fb01a2a90 con 0x7f2f9c0a3a60 2024-04-25T11:43:42.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.437+0000 7f2fa77fe640 1 -- 172.21.15.83:0/3710940726 <== mgr.15360 v2:172.21.15.83:6800/4112952368 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2fb01a2a90 con 0x7f2f9c0a3a60 2024-04-25T11:43:42.425 INFO:teuthology.orchestra.run.smithi083.stdout:/ 1m 6m3h 2024-04-25T11:43:42.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.439+0000 7f2fb4b43640 1 -- 172.21.15.83:0/3710940726 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f2f9c0a3a60 msgr2=0x7f2f9c0a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:42.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.439+0000 7f2fb4b43640 1 --2- 172.21.15.83:0/3710940726 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f2f9c0a3a60 0x7f2f9c0a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2f98000c00 tx=0x7f2f980314a0 comp rx=0 tx=0).stop 2024-04-25T11:43:42.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.439+0000 7f2fb4b43640 1 -- 172.21.15.83:0/3710940726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb0100030 msgr2=0x7f2fb019a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:43:42.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.439+0000 7f2fb4b43640 1 --2- 172.21.15.83:0/3710940726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb0100030 0x7f2fb019a350 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000e9c0 tx=0x7f2fa000ee80 comp rx=0 tx=0).stop 2024-04-25T11:43:42.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.439+0000 7f2fb4b43640 1 -- 172.21.15.83:0/3710940726 shutdown_connections 2024-04-25T11:43:42.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.439+0000 7f2fb4b43640 1 --2- 172.21.15.83:0/3710940726 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f2f9c0a3a60 0x7f2f9c0a5f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:42.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.439+0000 7f2fb4b43640 1 --2- 172.21.15.83:0/3710940726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fb01035c0 0x7f2fb01a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:42.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.439+0000 7f2fb4b43640 1 --2- 172.21.15.83:0/3710940726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2fb01009f0 0x7f2fb019a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:42.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.439+0000 7f2fb4b43640 1 --2- 172.21.15.83:0/3710940726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2fb0100030 0x7f2fb019a350 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:43:42.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.439+0000 7f2fb4b43640 1 -- 172.21.15.83:0/3710940726 >> 172.21.15.83:0/3710940726 conn(0x7f2fb00fbbb0 msgr2=0x7f2fb0108b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:43:42.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.440+0000 7f2fb4b43640 1 -- 172.21.15.83:0/3710940726 shutdown_connections 2024-04-25T11:43:42.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:43:42.440+0000 7f2fb4b43640 1 -- 172.21.15.83:0/3710940726 wait complete. 2024-04-25T11:43:42.438 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T11:43:42.463 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T11:43:42.473 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T11:43:42.489 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T11:43:42.489 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T11:43:42.505 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T11:43:42.516 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa/tasks/quiescer.py", line 386, in task cluster_name = ctx.config['cluster'] KeyError: 'cluster' 2024-04-25T11:43:42.745 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=f0b1f6da84f84f1bb872e8b74a7eb21f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa/tasks/quiescer.py", line 386, in task cluster_name = ctx.config['cluster'] KeyError: 'cluster' 2024-04-25T11:43:42.746 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T11:43:42.756 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T11:43:42.764 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T11:43:42.776 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T11:43:42.787 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T11:43:42.787 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T11:43:42.787 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T11:43:42.788 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T11:43:42.841 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T11:43:43.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:42 smithi083 ceph-mon[29330]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:43.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:42 smithi083 ceph-mon[29330]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:43:43.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:42 smithi083 ceph-mon[29330]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:43.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:42 smithi083 ceph-mon[29330]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:43.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:42 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:42 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:42 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:42 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:42 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:42 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:42 smithi094 ceph-mon[33032]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:43.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:42 smithi094 ceph-mon[33032]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:43:43.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:42 smithi094 ceph-mon[33032]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:43.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:42 smithi094 ceph-mon[33032]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:43.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:42 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:42 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:42 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:42 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:42 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:42 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:42 smithi111 ceph-mon[32784]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:43.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:42 smithi111 ceph-mon[32784]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:43:43.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:42 smithi111 ceph-mon[32784]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:43.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:42 smithi111 ceph-mon[32784]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:43.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:42 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:42 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:42 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:42 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:42 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:43.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:42 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:43:44.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:44.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T11:43:44.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:43:44.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:43 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:43:44.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:43:44.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:44.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T11:43:44.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:43:44.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:43 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:43:44.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:43:44.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:43:44.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T11:43:44.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:44.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:43:44.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:43:44.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:43 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:43:45.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-25T11:43:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-25T11:43:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi111 to 176.2M 2024-04-25T11:43:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: Unable to set osd_memory_target on smithi094 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T11:43:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T11:43:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: Unable to set osd_memory_target on smithi111 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-25T11:43:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-25T11:43:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-25T11:43:45.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-25T11:43:45.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: Updating smithi083:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:43:45.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: Updating smithi094:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:43:45.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: Updating smithi111:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:43:45.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:43:45.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:43:45.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:43:45.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:43:45.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:43:45.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:43:45.276 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:44 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-25T11:43:45.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-25T11:43:45.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi111 to 176.2M 2024-04-25T11:43:45.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: Unable to set osd_memory_target on smithi094 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T11:43:45.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T11:43:45.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: Unable to set osd_memory_target on smithi111 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-25T11:43:45.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-25T11:43:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-25T11:43:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-25T11:43:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: Updating smithi083:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:43:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: Updating smithi094:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:43:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: Updating smithi111:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:43:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:43:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:43:45.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:43:45.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:43:45.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:43:45.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:43:45.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:44 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-25T11:43:45.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-25T11:43:45.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi111 to 176.2M 2024-04-25T11:43:45.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: Unable to set osd_memory_target on smithi094 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T11:43:45.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T11:43:45.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: Unable to set osd_memory_target on smithi111 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-25T11:43:45.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-25T11:43:45.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-25T11:43:45.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-25T11:43:45.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: Updating smithi083:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:43:45.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: Updating smithi094:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:43:45.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: Updating smithi111:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T11:43:45.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:43:45.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:43:45.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-25T11:43:45.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:45.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:43:45.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:43:45.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:43:45.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:44 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:43:46.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:45 smithi094 ceph-mon[33032]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:43:46.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:45 smithi094 ceph-mon[33032]: Updating smithi083:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:43:46.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:45 smithi094 ceph-mon[33032]: Updating smithi111:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:43:46.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:45 smithi094 ceph-mon[33032]: Updating smithi094:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:43:46.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:45 smithi094 ceph-mon[33032]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T11:43:46.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:45 smithi111 ceph-mon[32784]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:43:46.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:45 smithi111 ceph-mon[32784]: Updating smithi083:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:43:46.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:45 smithi111 ceph-mon[32784]: Updating smithi111:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:43:46.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:45 smithi111 ceph-mon[32784]: Updating smithi094:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:43:46.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:45 smithi111 ceph-mon[32784]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T11:43:46.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:45 smithi083 ceph-mon[29330]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T11:43:46.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:45 smithi083 ceph-mon[29330]: Updating smithi083:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:43:46.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:45 smithi083 ceph-mon[29330]: Updating smithi111:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:43:46.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:45 smithi083 ceph-mon[29330]: Updating smithi094:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T11:43:46.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:45 smithi083 ceph-mon[29330]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T11:43:48.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:47 smithi094 ceph-mon[33032]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 90 KiB/s wr, 66 op/s 2024-04-25T11:43:48.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:47 smithi111 ceph-mon[32784]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 90 KiB/s wr, 66 op/s 2024-04-25T11:43:48.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:47 smithi083 ceph-mon[29330]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 90 KiB/s wr, 66 op/s 2024-04-25T11:43:50.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:49 smithi094 ceph-mon[33032]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 131 KiB/s wr, 78 op/s 2024-04-25T11:43:50.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:49 smithi111 ceph-mon[32784]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 131 KiB/s wr, 78 op/s 2024-04-25T11:43:50.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:49 smithi083 ceph-mon[29330]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 131 KiB/s wr, 78 op/s 2024-04-25T11:43:52.206 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:51 smithi083 ceph-mon[29330]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-04-25T11:43:52.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:51 smithi094 ceph-mon[33032]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-04-25T11:43:52.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:51 smithi111 ceph-mon[32784]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-04-25T11:43:54.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:53 smithi094 ceph-mon[33032]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-04-25T11:43:54.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:53 smithi111 ceph-mon[32784]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-04-25T11:43:54.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:53 smithi083 ceph-mon[29330]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-04-25T11:43:56.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:55 smithi094 ceph-mon[33032]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T11:43:56.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:55 smithi111 ceph-mon[32784]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T11:43:56.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:55 smithi083 ceph-mon[29330]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-04-25T11:43:58.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:57 smithi094 ceph-mon[33032]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-04-25T11:43:58.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:57 smithi111 ceph-mon[32784]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-04-25T11:43:58.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:57 smithi083 ceph-mon[29330]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-04-25T11:44:00.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:43:59 smithi094 ceph-mon[33032]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 39 KiB/s wr, 26 op/s 2024-04-25T11:44:00.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:43:59 smithi111 ceph-mon[32784]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 39 KiB/s wr, 26 op/s 2024-04-25T11:44:00.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:43:59 smithi083 ceph-mon[29330]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 39 KiB/s wr, 26 op/s 2024-04-25T11:44:01.155 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:44:00 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:44:00.731+0000 7efe0164e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:44:01.155 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:44:00 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:44:00.731+0000 7efe0164e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:44:01.155 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:44:00 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:44:00.731+0000 7efe0164e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:44:01.155 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:44:00 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:44:00.731+0000 7efe0164e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:44:01.155 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:44:00 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:44:00.731+0000 7efe0164e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T11:44:02.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:01 smithi094 ceph-mon[33032]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:02.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:01 smithi111 ceph-mon[32784]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:02.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:01 smithi083 ceph-mon[29330]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:04.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:03 smithi094 ceph-mon[33032]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:44:04.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:03 smithi094 ceph-mon[33032]: mgrmap e25: x(active, since 22s), standbys: z, y 2024-04-25T11:44:04.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:03 smithi111 ceph-mon[32784]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:44:04.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:03 smithi111 ceph-mon[32784]: mgrmap e25: x(active, since 22s), standbys: z, y 2024-04-25T11:44:04.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:03 smithi083 ceph-mon[29330]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:44:04.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:03 smithi083 ceph-mon[29330]: mgrmap e25: x(active, since 22s), standbys: z, y 2024-04-25T11:44:06.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:05 smithi094 ceph-mon[33032]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:44:06.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:05 smithi111 ceph-mon[32784]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:44:06.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:05 smithi083 ceph-mon[29330]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:44:08.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:07 smithi094 ceph-mon[33032]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T11:44:08.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:07 smithi111 ceph-mon[32784]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T11:44:08.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:07 smithi083 ceph-mon[29330]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-25T11:44:10.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:09 smithi094 ceph-mon[33032]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:44:10.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:09 smithi111 ceph-mon[32784]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:44:10.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:09 smithi083 ceph-mon[29330]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:44:12.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:11 smithi094 ceph-mon[33032]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:44:12.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:11 smithi111 ceph-mon[32784]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:44:12.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:11 smithi083 ceph-mon[29330]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:44:14.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:13 smithi094 ceph-mon[33032]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-25T11:44:14.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:13 smithi111 ceph-mon[32784]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-25T11:44:14.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:13 smithi083 ceph-mon[29330]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-25T11:44:16.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:15 smithi094 ceph-mon[33032]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T11:44:16.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:15 smithi111 ceph-mon[32784]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T11:44:16.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:15 smithi083 ceph-mon[29330]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T11:44:18.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:17 smithi094 ceph-mon[33032]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T11:44:18.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:17 smithi111 ceph-mon[32784]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T11:44:18.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:17 smithi083 ceph-mon[29330]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-25T11:44:20.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:19 smithi094 ceph-mon[33032]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-25T11:44:20.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:19 smithi111 ceph-mon[32784]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-25T11:44:20.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:19 smithi083 ceph-mon[29330]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-25T11:44:22.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:21 smithi094 ceph-mon[33032]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:44:22.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:21 smithi111 ceph-mon[32784]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:44:22.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:21 smithi083 ceph-mon[29330]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:44:24.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:23 smithi094 ceph-mon[33032]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T11:44:24.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:23 smithi111 ceph-mon[32784]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T11:44:24.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:23 smithi083 ceph-mon[29330]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T11:44:26.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:25 smithi094 ceph-mon[33032]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:26.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:25 smithi111 ceph-mon[32784]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:26.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:25 smithi083 ceph-mon[29330]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:28.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:28 smithi094 ceph-mon[33032]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:28.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:28 smithi111 ceph-mon[32784]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:28.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:28 smithi083 ceph-mon[29330]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:30.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:30 smithi094 ceph-mon[33032]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T11:44:30.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:30 smithi111 ceph-mon[32784]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T11:44:30.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:30 smithi083 ceph-mon[29330]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-04-25T11:44:32.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:32 smithi094 ceph-mon[33032]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:32.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:32 smithi111 ceph-mon[32784]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:32.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:32 smithi083 ceph-mon[29330]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:34.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:34 smithi094 ceph-mon[33032]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:34.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:34 smithi111 ceph-mon[32784]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:34.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:34 smithi083 ceph-mon[29330]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:36.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:36 smithi094 ceph-mon[33032]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:36.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:36 smithi111 ceph-mon[32784]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:36.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:36 smithi083 ceph-mon[29330]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:38.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:38 smithi094 ceph-mon[33032]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:38.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:38 smithi111 ceph-mon[32784]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:38.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:38 smithi083 ceph-mon[29330]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:40.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:40 smithi094 ceph-mon[33032]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:40.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:40 smithi111 ceph-mon[32784]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:40.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:40 smithi083 ceph-mon[29330]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:42.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:42 smithi094 ceph-mon[33032]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:42.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:42 smithi111 ceph-mon[32784]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:42.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:42 smithi083 ceph-mon[29330]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:44.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:44 smithi094 ceph-mon[33032]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:44.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:44 smithi111 ceph-mon[32784]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:44.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:44 smithi083 ceph-mon[29330]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:45.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:45 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:44:45.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:45 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:44:45.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:45 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:44:46.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:46 smithi094 ceph-mon[33032]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:46.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:46 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:44:46.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:46 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:44:46.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:46 smithi111 ceph-mon[32784]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:46.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:46 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:44:46.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:46 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:44:46.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:46 smithi083 ceph-mon[29330]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:46.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:46 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:44:46.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:46 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:44:47.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:47 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:44:47.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:47 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:44:47.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:47 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:44:47.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:47 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:44:47.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:47 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:44:47.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:47 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:44:47.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:47 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:44:47.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:47 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:44:47.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:47 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:44:48.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:48 smithi111 ceph-mon[32784]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:48.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:48 smithi083 ceph-mon[29330]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:48.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:48 smithi094 ceph-mon[33032]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:50.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:50 smithi111 ceph-mon[32784]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:50.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:50 smithi083 ceph-mon[29330]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:50.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:50 smithi094 ceph-mon[33032]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:52.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:52 smithi111 ceph-mon[32784]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:52.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:52 smithi083 ceph-mon[29330]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:52.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:52 smithi094 ceph-mon[33032]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:54.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:54 smithi111 ceph-mon[32784]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:54.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:54 smithi083 ceph-mon[29330]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:54.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:54 smithi094 ceph-mon[33032]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:56.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:56 smithi111 ceph-mon[32784]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:56.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:56 smithi083 ceph-mon[29330]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:56.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:56 smithi094 ceph-mon[33032]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:44:58.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:44:58 smithi111 ceph-mon[32784]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:58.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:44:58 smithi083 ceph-mon[29330]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:44:58.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:44:58 smithi094 ceph-mon[33032]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:00.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:00 smithi111 ceph-mon[32784]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:00.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:00 smithi083 ceph-mon[29330]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:00.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:00 smithi094 ceph-mon[33032]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:02.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:02 smithi111 ceph-mon[32784]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:02.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:02 smithi083 ceph-mon[29330]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:02.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:02 smithi094 ceph-mon[33032]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:04.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:04 smithi111 ceph-mon[32784]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:45:04.324 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:04 smithi094 ceph-mon[33032]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:45:04.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:04 smithi083 ceph-mon[29330]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:45:06.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:06 smithi111 ceph-mon[32784]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:45:06.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:06 smithi083 ceph-mon[29330]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:45:06.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:06 smithi094 ceph-mon[33032]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:45:08.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:08 smithi111 ceph-mon[32784]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:45:08.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:08 smithi083 ceph-mon[29330]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:45:08.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:08 smithi094 ceph-mon[33032]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:45:10.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:10 smithi111 ceph-mon[32784]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:45:10.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:10 smithi083 ceph-mon[29330]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:45:10.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:10 smithi094 ceph-mon[33032]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:45:12.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:12 smithi111 ceph-mon[32784]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:45:12.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:12 smithi083 ceph-mon[29330]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:45:12.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:12 smithi094 ceph-mon[33032]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:45:14.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:14 smithi083 ceph-mon[29330]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-25T11:45:14.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:14 smithi083 ceph-mon[29330]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T11:45:14.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:14 smithi094 ceph-mon[33032]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-25T11:45:14.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:14 smithi094 ceph-mon[33032]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T11:45:14.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:14 smithi111 ceph-mon[32784]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-25T11:45:14.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:14 smithi111 ceph-mon[32784]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T11:45:16.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:16 smithi083 ceph-mon[29330]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T11:45:16.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:16 smithi094 ceph-mon[33032]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T11:45:16.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:16 smithi111 ceph-mon[32784]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T11:45:18.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:18 smithi083 ceph-mon[29330]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-25T11:45:18.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:18 smithi094 ceph-mon[33032]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-25T11:45:18.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:18 smithi111 ceph-mon[32784]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-25T11:45:20.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:20 smithi083 ceph-mon[29330]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T11:45:20.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:20 smithi094 ceph-mon[33032]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T11:45:20.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:20 smithi111 ceph-mon[32784]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T11:45:22.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:22 smithi083 ceph-mon[29330]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T11:45:22.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:22 smithi094 ceph-mon[33032]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T11:45:22.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:22 smithi111 ceph-mon[32784]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T11:45:24.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:24 smithi083 ceph-mon[29330]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-25T11:45:24.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:24 smithi094 ceph-mon[33032]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-25T11:45:24.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:24 smithi111 ceph-mon[32784]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-25T11:45:26.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:26 smithi083 ceph-mon[29330]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:26.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:26 smithi094 ceph-mon[33032]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:26.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:26 smithi111 ceph-mon[32784]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:28.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:28 smithi083 ceph-mon[29330]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:28.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:28 smithi094 ceph-mon[33032]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:28.573 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:28 smithi111 ceph-mon[32784]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:30.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:30 smithi083 ceph-mon[29330]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:30.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:30 smithi094 ceph-mon[33032]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:30.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:30 smithi111 ceph-mon[32784]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:32.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:32 smithi083 ceph-mon[29330]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:32.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:32 smithi094 ceph-mon[33032]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:32.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:32 smithi111 ceph-mon[32784]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:34.373 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:34 smithi111 ceph-mon[32784]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:34.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:34 smithi083 ceph-mon[29330]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:34.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:34 smithi094 ceph-mon[33032]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:36.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:36 smithi083 ceph-mon[29330]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:36.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:36 smithi094 ceph-mon[33032]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:36.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:36 smithi111 ceph-mon[32784]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:38.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:38 smithi083 ceph-mon[29330]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:38.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:38 smithi094 ceph-mon[33032]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:38.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:38 smithi111 ceph-mon[32784]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:40.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:40 smithi083 ceph-mon[29330]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:40.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:40 smithi094 ceph-mon[33032]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:40.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:40 smithi111 ceph-mon[32784]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:42.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:42 smithi083 ceph-mon[29330]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:42.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:42 smithi094 ceph-mon[33032]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:42.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:42 smithi111 ceph-mon[32784]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:44.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:44 smithi083 ceph-mon[29330]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:44.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:44 smithi094 ceph-mon[33032]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:44.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:44 smithi111 ceph-mon[32784]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:46.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:46 smithi083 ceph-mon[29330]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:46.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:46 smithi094 ceph-mon[33032]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:46.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:46 smithi111 ceph-mon[32784]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:47.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:47 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:45:47.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:47 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:45:47.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:47 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:45:47.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:47 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:45:47.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:47 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:45:47.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:47 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:45:47.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:47 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:45:47.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:47 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:45:47.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:47 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:45:47.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:47 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:45:47.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:47 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:45:47.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:47 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:45:47.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:47 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:45:47.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:47 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:45:47.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:47 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:45:48.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:48 smithi083 ceph-mon[29330]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:48.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:48 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:45:48.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:48 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:45:48.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:48 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:45:48.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:48 smithi094 ceph-mon[33032]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:48.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:48 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:45:48.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:48 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:45:48.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:48 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:45:48.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:48 smithi111 ceph-mon[32784]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:48.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:48 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:45:48.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:48 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:45:48.571 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:48 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:45:49.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:49 smithi083 ceph-mon[29330]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:49.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:49 smithi094 ceph-mon[33032]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:49.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:49 smithi111 ceph-mon[32784]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:51.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:51 smithi111 ceph-mon[32784]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:51.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:51 smithi083 ceph-mon[29330]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:51.772 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:51 smithi094 ceph-mon[33032]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:53.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:53 smithi111 ceph-mon[32784]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:53.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:53 smithi083 ceph-mon[29330]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:53.771 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:53 smithi094 ceph-mon[33032]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:55.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:55 smithi111 ceph-mon[32784]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:55.654 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:55 smithi083 ceph-mon[29330]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:55.771 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:55 smithi094 ceph-mon[33032]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:45:57.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:57 smithi083 ceph-mon[29330]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:58.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:57 smithi094 ceph-mon[33032]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:58.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:57 smithi111 ceph-mon[32784]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:45:59.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:45:59 smithi083 ceph-mon[29330]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:00.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:45:59 smithi094 ceph-mon[33032]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:00.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:45:59 smithi111 ceph-mon[32784]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:01.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:01 smithi083 ceph-mon[29330]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:02.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:01 smithi094 ceph-mon[33032]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:02.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:01 smithi111 ceph-mon[32784]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:03.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:03 smithi083 ceph-mon[29330]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:46:04.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:03 smithi094 ceph-mon[33032]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:46:04.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:03 smithi111 ceph-mon[32784]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:46:05.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:05 smithi083 ceph-mon[29330]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:46:06.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:05 smithi094 ceph-mon[33032]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:46:06.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:05 smithi111 ceph-mon[32784]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:46:07.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:07 smithi083 ceph-mon[29330]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:46:08.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:07 smithi094 ceph-mon[33032]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:46:08.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:07 smithi111 ceph-mon[32784]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:46:09.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:09 smithi083 ceph-mon[29330]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:46:10.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:09 smithi094 ceph-mon[33032]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:46:10.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:09 smithi111 ceph-mon[32784]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:46:11.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:11 smithi083 ceph-mon[29330]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:46:12.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:11 smithi094 ceph-mon[33032]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:46:12.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:11 smithi111 ceph-mon[32784]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:46:13.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:13 smithi083 ceph-mon[29330]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:46:14.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:13 smithi094 ceph-mon[33032]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:46:14.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:13 smithi111 ceph-mon[32784]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:46:15.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:15 smithi083 ceph-mon[29330]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T11:46:16.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:15 smithi094 ceph-mon[33032]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T11:46:16.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:15 smithi111 ceph-mon[32784]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T11:46:17.904 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:17 smithi083 ceph-mon[29330]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-25T11:46:18.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:17 smithi094 ceph-mon[33032]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-25T11:46:18.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:17 smithi111 ceph-mon[32784]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-25T11:46:20.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:19 smithi094 ceph-mon[33032]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:46:20.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:19 smithi111 ceph-mon[32784]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:46:20.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:19 smithi083 ceph-mon[29330]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:46:22.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:21 smithi094 ceph-mon[33032]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:46:22.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:21 smithi111 ceph-mon[32784]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:46:22.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:21 smithi083 ceph-mon[29330]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:46:23.957 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:23 smithi083 ceph-mon[29330]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T11:46:24.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:23 smithi094 ceph-mon[33032]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T11:46:24.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:23 smithi111 ceph-mon[32784]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T11:46:26.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:25 smithi094 ceph-mon[33032]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:26.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:25 smithi111 ceph-mon[32784]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:26.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:25 smithi083 ceph-mon[29330]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:28.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:27 smithi094 ceph-mon[33032]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:28.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:27 smithi111 ceph-mon[32784]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:28.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:27 smithi083 ceph-mon[29330]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:30.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:29 smithi094 ceph-mon[33032]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:30.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:29 smithi111 ceph-mon[32784]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:30.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:29 smithi083 ceph-mon[29330]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:32.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:31 smithi094 ceph-mon[33032]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:32.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:31 smithi111 ceph-mon[32784]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:32.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:31 smithi083 ceph-mon[29330]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:34.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:33 smithi094 ceph-mon[33032]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:34.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:33 smithi111 ceph-mon[32784]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:34.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:33 smithi083 ceph-mon[29330]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:36.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:35 smithi094 ceph-mon[33032]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:36.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:35 smithi111 ceph-mon[32784]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:36.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:35 smithi083 ceph-mon[29330]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:38.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:37 smithi094 ceph-mon[33032]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:38.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:37 smithi111 ceph-mon[32784]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:38.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:37 smithi083 ceph-mon[29330]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:40.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:39 smithi094 ceph-mon[33032]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:40.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:39 smithi111 ceph-mon[32784]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:40.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:39 smithi083 ceph-mon[29330]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:42.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:41 smithi094 ceph-mon[33032]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:42.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:41 smithi111 ceph-mon[32784]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:42.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:41 smithi083 ceph-mon[29330]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:44.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:43 smithi094 ceph-mon[33032]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:44.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:43 smithi111 ceph-mon[32784]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:44.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:43 smithi083 ceph-mon[29330]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:46.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:45 smithi094 ceph-mon[33032]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:46.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:45 smithi111 ceph-mon[32784]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:46.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:45 smithi083 ceph-mon[29330]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:48.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:47 smithi094 ceph-mon[33032]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:48.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:47 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:46:48.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:47 smithi111 ceph-mon[32784]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:48.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:47 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:46:48.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:47 smithi083 ceph-mon[29330]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:48.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:47 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:46:49.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:48 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:46:49.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:48 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:46:49.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:48 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:46:49.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:48 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:46:49.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:48 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:46:49.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:48 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:46:49.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:48 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:46:49.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:48 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:46:49.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:48 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:46:50.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:49 smithi094 ceph-mon[33032]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:50.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:49 smithi111 ceph-mon[32784]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:50.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:49 smithi083 ceph-mon[29330]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:52.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:51 smithi094 ceph-mon[33032]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:52.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:51 smithi111 ceph-mon[32784]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:52.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:51 smithi083 ceph-mon[29330]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:54.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:53 smithi094 ceph-mon[33032]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:54.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:53 smithi111 ceph-mon[32784]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:54.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:53 smithi083 ceph-mon[29330]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:56.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:55 smithi094 ceph-mon[33032]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:56.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:55 smithi111 ceph-mon[32784]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:56.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:55 smithi083 ceph-mon[29330]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:46:58.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:57 smithi094 ceph-mon[33032]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:58.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:57 smithi111 ceph-mon[32784]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:46:58.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:57 smithi083 ceph-mon[29330]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:00.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:46:59 smithi094 ceph-mon[33032]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:00.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:46:59 smithi111 ceph-mon[32784]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:00.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:46:59 smithi083 ceph-mon[29330]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:02.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:01 smithi094 ceph-mon[33032]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:02.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:01 smithi111 ceph-mon[32784]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:02.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:01 smithi083 ceph-mon[29330]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:04.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:03 smithi094 ceph-mon[33032]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:47:04.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:03 smithi111 ceph-mon[32784]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:47:04.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:03 smithi083 ceph-mon[29330]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:47:06.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:05 smithi094 ceph-mon[33032]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:47:06.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:05 smithi111 ceph-mon[32784]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:47:06.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:05 smithi083 ceph-mon[29330]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:47:08.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:07 smithi094 ceph-mon[33032]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:47:08.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:07 smithi111 ceph-mon[32784]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:47:08.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:07 smithi083 ceph-mon[29330]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:47:10.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:09 smithi094 ceph-mon[33032]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:47:10.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:09 smithi111 ceph-mon[32784]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:47:10.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:09 smithi083 ceph-mon[29330]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:47:12.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:11 smithi094 ceph-mon[33032]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:47:12.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:11 smithi111 ceph-mon[32784]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:47:12.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:11 smithi083 ceph-mon[29330]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:47:14.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:13 smithi094 ceph-mon[33032]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:47:14.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:13 smithi111 ceph-mon[32784]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:47:14.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:13 smithi083 ceph-mon[29330]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:47:16.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:15 smithi094 ceph-mon[33032]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T11:47:16.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:15 smithi111 ceph-mon[32784]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T11:47:16.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:15 smithi083 ceph-mon[29330]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T11:47:18.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:17 smithi094 ceph-mon[33032]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-25T11:47:18.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:17 smithi111 ceph-mon[32784]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-25T11:47:18.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:17 smithi083 ceph-mon[29330]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-25T11:47:20.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:19 smithi094 ceph-mon[33032]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:47:20.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:19 smithi111 ceph-mon[32784]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:47:20.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:19 smithi083 ceph-mon[29330]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:47:22.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:21 smithi094 ceph-mon[33032]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:47:22.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:21 smithi111 ceph-mon[32784]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:47:22.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:21 smithi083 ceph-mon[29330]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:47:24.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:23 smithi094 ceph-mon[33032]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T11:47:24.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:23 smithi111 ceph-mon[32784]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T11:47:24.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:23 smithi083 ceph-mon[29330]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T11:47:26.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:25 smithi094 ceph-mon[33032]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:26.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:25 smithi111 ceph-mon[32784]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:26.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:25 smithi083 ceph-mon[29330]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:28.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:27 smithi094 ceph-mon[33032]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:28.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:27 smithi111 ceph-mon[32784]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:28.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:27 smithi083 ceph-mon[29330]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:30.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:29 smithi094 ceph-mon[33032]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:30.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:29 smithi111 ceph-mon[32784]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:30.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:29 smithi083 ceph-mon[29330]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:32.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:31 smithi094 ceph-mon[33032]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:32.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:31 smithi111 ceph-mon[32784]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:32.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:31 smithi083 ceph-mon[29330]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:34.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:33 smithi111 ceph-mon[32784]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:34.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:33 smithi083 ceph-mon[29330]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:34.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:33 smithi094 ceph-mon[33032]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:36.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:35 smithi111 ceph-mon[32784]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:36.072 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:35 smithi094 ceph-mon[33032]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:36.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:35 smithi083 ceph-mon[29330]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:38.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:37 smithi111 ceph-mon[32784]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:38.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:37 smithi083 ceph-mon[29330]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:38.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:37 smithi094 ceph-mon[33032]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:40.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:39 smithi111 ceph-mon[32784]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:40.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:39 smithi083 ceph-mon[29330]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:40.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:39 smithi094 ceph-mon[33032]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:42.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:41 smithi111 ceph-mon[32784]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:42.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:41 smithi083 ceph-mon[29330]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:42.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:41 smithi094 ceph-mon[33032]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:44.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:43 smithi111 ceph-mon[32784]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:44.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:43 smithi083 ceph-mon[29330]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:44.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:43 smithi094 ceph-mon[33032]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:46.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:45 smithi111 ceph-mon[32784]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:46.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:45 smithi083 ceph-mon[29330]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:46.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:45 smithi094 ceph-mon[33032]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:48.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:47 smithi111 ceph-mon[32784]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:48.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:47 smithi083 ceph-mon[29330]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:48.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:47 smithi094 ceph-mon[33032]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:49.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:48 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:47:49.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:48 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:47:49.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:48 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:47:50.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:49 smithi111 ceph-mon[32784]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:50.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:49 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:47:50.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:49 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:47:50.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:49 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:47:50.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:49 smithi083 ceph-mon[29330]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:50.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:49 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:47:50.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:49 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:47:50.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:49 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:47:50.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:49 smithi094 ceph-mon[33032]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:50.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:49 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:47:50.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:49 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:47:50.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:49 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:47:52.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:51 smithi111 ceph-mon[32784]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:52.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:51 smithi083 ceph-mon[29330]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:52.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:51 smithi094 ceph-mon[33032]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:54.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:53 smithi111 ceph-mon[32784]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:54.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:53 smithi083 ceph-mon[29330]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:54.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:53 smithi094 ceph-mon[33032]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:56.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:55 smithi111 ceph-mon[32784]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:56.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:55 smithi083 ceph-mon[29330]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:56.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:55 smithi094 ceph-mon[33032]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:47:58.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:57 smithi111 ceph-mon[32784]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:58.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:57 smithi083 ceph-mon[29330]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:47:58.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:57 smithi094 ceph-mon[33032]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:00.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:47:59 smithi111 ceph-mon[32784]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:00.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:47:59 smithi083 ceph-mon[29330]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:00.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:47:59 smithi094 ceph-mon[33032]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:02.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:01 smithi083 ceph-mon[29330]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:02.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:01 smithi094 ceph-mon[33032]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:02.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:01 smithi111 ceph-mon[32784]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:04.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:03 smithi083 ceph-mon[29330]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:48:04.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:03 smithi094 ceph-mon[33032]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:48:04.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:03 smithi111 ceph-mon[32784]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:48:06.122 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:05 smithi111 ceph-mon[32784]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:48:06.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:05 smithi083 ceph-mon[29330]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:48:06.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:05 smithi094 ceph-mon[33032]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:48:08.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:07 smithi083 ceph-mon[29330]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:48:08.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:07 smithi094 ceph-mon[33032]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:48:08.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:07 smithi111 ceph-mon[32784]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:48:10.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:09 smithi083 ceph-mon[29330]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:48:10.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:09 smithi094 ceph-mon[33032]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:48:10.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:09 smithi111 ceph-mon[32784]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:48:12.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:11 smithi083 ceph-mon[29330]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:48:12.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:11 smithi094 ceph-mon[33032]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:48:12.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:11 smithi111 ceph-mon[32784]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:48:14.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:13 smithi083 ceph-mon[29330]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:48:14.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:13 smithi094 ceph-mon[33032]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:48:14.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:13 smithi111 ceph-mon[32784]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:48:16.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:15 smithi083 ceph-mon[29330]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:48:16.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:15 smithi094 ceph-mon[33032]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:48:16.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:15 smithi111 ceph-mon[32784]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:48:18.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:17 smithi083 ceph-mon[29330]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T11:48:18.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:17 smithi094 ceph-mon[33032]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T11:48:18.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:17 smithi111 ceph-mon[32784]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T11:48:20.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:19 smithi083 ceph-mon[29330]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:20.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:19 smithi094 ceph-mon[33032]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:20.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:19 smithi111 ceph-mon[32784]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:22.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:21 smithi083 ceph-mon[29330]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:22.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:21 smithi094 ceph-mon[33032]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:22.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:21 smithi111 ceph-mon[32784]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:24.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:23 smithi083 ceph-mon[29330]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:24.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:23 smithi094 ceph-mon[33032]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:24.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:23 smithi111 ceph-mon[32784]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:26.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:25 smithi083 ceph-mon[29330]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:26.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:25 smithi094 ceph-mon[33032]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:26.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:25 smithi111 ceph-mon[32784]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:28.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:27 smithi083 ceph-mon[29330]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:28.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:27 smithi094 ceph-mon[33032]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:28.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:27 smithi111 ceph-mon[32784]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:30.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:29 smithi083 ceph-mon[29330]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:30.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:29 smithi094 ceph-mon[33032]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:30.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:29 smithi111 ceph-mon[32784]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:32.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:31 smithi083 ceph-mon[29330]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:32.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:31 smithi094 ceph-mon[33032]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:32.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:31 smithi111 ceph-mon[32784]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:34.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:33 smithi083 ceph-mon[29330]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:34.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:33 smithi094 ceph-mon[33032]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:34.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:33 smithi111 ceph-mon[32784]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:36.122 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:35 smithi111 ceph-mon[32784]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:36.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:35 smithi083 ceph-mon[29330]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:36.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:35 smithi094 ceph-mon[33032]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:38.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:37 smithi083 ceph-mon[29330]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:38.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:37 smithi094 ceph-mon[33032]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:38.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:37 smithi111 ceph-mon[32784]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:40.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:39 smithi083 ceph-mon[29330]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:40.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:39 smithi094 ceph-mon[33032]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:40.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:39 smithi111 ceph-mon[32784]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:42.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:41 smithi083 ceph-mon[29330]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:42.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:41 smithi094 ceph-mon[33032]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:42.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:41 smithi111 ceph-mon[32784]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:44.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:43 smithi083 ceph-mon[29330]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:44.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:43 smithi094 ceph-mon[33032]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:44.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:43 smithi111 ceph-mon[32784]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:46.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:45 smithi083 ceph-mon[29330]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:46.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:45 smithi094 ceph-mon[33032]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:46.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:45 smithi111 ceph-mon[32784]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:48.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:47 smithi083 ceph-mon[29330]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:48.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:47 smithi094 ceph-mon[33032]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:48.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:47 smithi111 ceph-mon[32784]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:50.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:49 smithi083 ceph-mon[29330]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:50.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:49 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:48:50.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:49 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:48:50.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:49 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:48:50.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:49 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:48:50.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:49 smithi094 ceph-mon[33032]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:50.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:49 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:48:50.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:49 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:48:50.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:49 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:48:50.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:49 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:48:50.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:49 smithi111 ceph-mon[32784]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:50.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:49 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:48:50.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:49 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:48:50.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:49 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:48:50.321 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:49 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:48:52.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:51 smithi083 ceph-mon[29330]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:52.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:51 smithi094 ceph-mon[33032]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:52.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:51 smithi111 ceph-mon[32784]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:54.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:53 smithi083 ceph-mon[29330]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:54.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:53 smithi094 ceph-mon[33032]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:54.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:53 smithi111 ceph-mon[32784]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:56.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:55 smithi083 ceph-mon[29330]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:56.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:55 smithi094 ceph-mon[33032]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:56.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:55 smithi111 ceph-mon[32784]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:48:58.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:57 smithi094 ceph-mon[33032]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:58.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:57 smithi111 ceph-mon[32784]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:48:58.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:57 smithi083 ceph-mon[29330]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:49:00.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:48:59 smithi094 ceph-mon[33032]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:49:00.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:48:59 smithi111 ceph-mon[32784]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:49:00.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:48:59 smithi083 ceph-mon[29330]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:49:02.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:01 smithi094 ceph-mon[33032]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:49:02.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:01 smithi111 ceph-mon[32784]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:49:02.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:01 smithi083 ceph-mon[29330]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:49:04.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:03 smithi094 ceph-mon[33032]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:49:04.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:03 smithi111 ceph-mon[32784]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:49:04.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:03 smithi083 ceph-mon[29330]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:49:06.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:05 smithi094 ceph-mon[33032]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:49:06.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:05 smithi111 ceph-mon[32784]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:49:06.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:05 smithi083 ceph-mon[29330]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:49:08.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:07 smithi094 ceph-mon[33032]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:49:08.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:07 smithi111 ceph-mon[32784]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:49:08.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:07 smithi083 ceph-mon[29330]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:49:10.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:09 smithi094 ceph-mon[33032]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:49:10.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:09 smithi111 ceph-mon[32784]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:49:10.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:09 smithi083 ceph-mon[29330]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:49:12.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:11 smithi094 ceph-mon[33032]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:49:12.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:11 smithi111 ceph-mon[32784]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:49:12.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:11 smithi083 ceph-mon[29330]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-25T11:49:14.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:13 smithi094 ceph-mon[33032]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:49:14.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:13 smithi111 ceph-mon[32784]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:49:14.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:13 smithi083 ceph-mon[29330]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-25T11:49:16.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:15 smithi094 ceph-mon[33032]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T11:49:16.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:15 smithi111 ceph-mon[32784]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T11:49:16.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:15 smithi083 ceph-mon[29330]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-25T11:49:18.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:17 smithi094 ceph-mon[33032]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-25T11:49:18.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:17 smithi111 ceph-mon[32784]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-25T11:49:18.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:17 smithi083 ceph-mon[29330]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-25T11:49:20.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:19 smithi094 ceph-mon[33032]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:49:20.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:19 smithi111 ceph-mon[32784]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:49:20.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:19 smithi083 ceph-mon[29330]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:49:22.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:21 smithi094 ceph-mon[33032]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:49:22.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:21 smithi111 ceph-mon[32784]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:49:22.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:21 smithi083 ceph-mon[29330]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-25T11:49:24.206 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:23 smithi083 ceph-mon[29330]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T11:49:24.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:23 smithi094 ceph-mon[33032]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T11:49:24.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:23 smithi111 ceph-mon[32784]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-25T11:49:26.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:25 smithi094 ceph-mon[33032]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:49:26.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:25 smithi111 ceph-mon[32784]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:49:26.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:25 smithi083 ceph-mon[29330]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:49:28.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:27 smithi094 ceph-mon[33032]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:49:28.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:27 smithi111 ceph-mon[32784]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:49:28.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:27 smithi083 ceph-mon[29330]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:49:30.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:29 smithi094 ceph-mon[33032]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:49:30.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:29 smithi111 ceph-mon[32784]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:49:30.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:29 smithi083 ceph-mon[29330]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:49:32.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:31 smithi094 ceph-mon[33032]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:49:32.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:31 smithi111 ceph-mon[32784]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:49:32.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:31 smithi083 ceph-mon[29330]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:49:34.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:33 smithi094 ceph-mon[33032]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:49:34.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:33 smithi111 ceph-mon[32784]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:49:34.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:33 smithi083 ceph-mon[29330]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:49:36.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:35 smithi094 ceph-mon[33032]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:49:36.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:35 smithi111 ceph-mon[32784]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:49:36.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:35 smithi083 ceph-mon[29330]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:49:38.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:37 smithi094 ceph-mon[33032]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:49:38.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:37 smithi111 ceph-mon[32784]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:49:38.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:37 smithi083 ceph-mon[29330]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:49:40.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:39 smithi094 ceph-mon[33032]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:49:40.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:39 smithi111 ceph-mon[32784]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:49:40.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:39 smithi083 ceph-mon[29330]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:49:42.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:41 smithi094 ceph-mon[33032]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:49:42.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:41 smithi111 ceph-mon[32784]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:49:42.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:41 smithi083 ceph-mon[29330]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:49:44.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:43 smithi094 ceph-mon[33032]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:49:44.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:43 smithi111 ceph-mon[32784]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:49:44.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:43 smithi083 ceph-mon[29330]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:49:46.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:45 smithi094 ceph-mon[33032]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:49:46.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:45 smithi111 ceph-mon[32784]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:49:46.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:45 smithi083 ceph-mon[29330]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:49:48.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:48 smithi094 ceph-mon[33032]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-25T11:49:48.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:48 smithi111 ceph-mon[32784]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-25T11:49:48.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:48 smithi083 ceph-mon[29330]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-25T11:49:50.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:50 smithi094 ceph-mon[33032]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-25T11:49:50.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:50 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:49:50.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:50 smithi111 ceph-mon[32784]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-25T11:49:50.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:50 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:49:50.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:50 smithi083 ceph-mon[29330]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-25T11:49:50.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:50 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:49:51.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:51 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:49:51.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:51 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:49:51.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:51 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:49:51.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:51 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:49:51.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:51 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:49:51.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:51 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:49:51.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:51 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:49:51.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:51 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:49:51.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:51 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:49:52.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:52 smithi094 ceph-mon[33032]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-25T11:49:52.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:52 smithi111 ceph-mon[32784]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-25T11:49:52.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:52 smithi083 ceph-mon[29330]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-04-25T11:49:54.270 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:54 smithi094 ceph-mon[33032]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-25T11:49:54.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:54 smithi111 ceph-mon[32784]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-25T11:49:54.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:54 smithi083 ceph-mon[29330]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-25T11:49:56.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:56 smithi094 ceph-mon[33032]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:49:56.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:56 smithi111 ceph-mon[32784]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:49:56.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:56 smithi083 ceph-mon[29330]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-25T11:49:58.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:49:58 smithi094 ceph-mon[33032]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:49:58.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:49:58 smithi111 ceph-mon[32784]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:49:58.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:49:58 smithi083 ceph-mon[29330]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:50:00.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:00 smithi094 ceph-mon[33032]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:00.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:00 smithi094 ceph-mon[33032]: overall HEALTH_OK 2024-04-25T11:50:00.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:00 smithi111 ceph-mon[32784]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:00.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:00 smithi111 ceph-mon[32784]: overall HEALTH_OK 2024-04-25T11:50:00.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:00 smithi083 ceph-mon[29330]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:00.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:00 smithi083 ceph-mon[29330]: overall HEALTH_OK 2024-04-25T11:50:02.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:02 smithi094 ceph-mon[33032]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:02.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:02 smithi111 ceph-mon[32784]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:02.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:02 smithi083 ceph-mon[29330]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:04.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:04 smithi094 ceph-mon[33032]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:50:04.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:04 smithi111 ceph-mon[32784]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:50:04.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:04 smithi083 ceph-mon[29330]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:50:06.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:06 smithi094 ceph-mon[33032]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:50:06.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:06 smithi111 ceph-mon[32784]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:50:06.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:06 smithi083 ceph-mon[29330]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:50:08.271 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:08 smithi094 ceph-mon[33032]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:50:08.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:08 smithi111 ceph-mon[32784]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:50:08.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:08 smithi083 ceph-mon[29330]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:50:10.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:10 smithi111 ceph-mon[32784]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:50:10.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:10 smithi083 ceph-mon[29330]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:50:10.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:10 smithi094 ceph-mon[33032]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:50:12.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:12 smithi111 ceph-mon[32784]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:50:12.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:12 smithi111 ceph-mon[32784]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T11:50:12.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:12 smithi083 ceph-mon[29330]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:50:12.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:12 smithi083 ceph-mon[29330]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T11:50:12.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:12 smithi094 ceph-mon[33032]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:50:12.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:12 smithi094 ceph-mon[33032]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T11:50:14.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:14 smithi111 ceph-mon[32784]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:50:14.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:14 smithi083 ceph-mon[29330]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:50:14.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:14 smithi094 ceph-mon[33032]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:50:16.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:16 smithi111 ceph-mon[32784]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:50:16.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:16 smithi083 ceph-mon[29330]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:50:16.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:16 smithi094 ceph-mon[33032]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:50:18.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:18 smithi111 ceph-mon[32784]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:50:18.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:18 smithi083 ceph-mon[29330]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:50:18.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:18 smithi094 ceph-mon[33032]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:50:20.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:20 smithi111 ceph-mon[32784]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:50:20.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:20 smithi083 ceph-mon[29330]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:50:20.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:20 smithi094 ceph-mon[33032]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:50:22.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:22 smithi111 ceph-mon[32784]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:50:22.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:22 smithi111 ceph-mon[32784]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T11:50:22.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:22 smithi083 ceph-mon[29330]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:50:22.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:22 smithi083 ceph-mon[29330]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T11:50:22.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:22 smithi094 ceph-mon[33032]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:50:22.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:22 smithi094 ceph-mon[33032]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T11:50:24.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:24 smithi111 ceph-mon[32784]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:50:24.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:24 smithi083 ceph-mon[29330]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:50:24.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:24 smithi094 ceph-mon[33032]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:50:26.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:26 smithi111 ceph-mon[32784]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:50:26.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:26 smithi083 ceph-mon[29330]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:50:26.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:26 smithi094 ceph-mon[33032]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:50:28.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:28 smithi111 ceph-mon[32784]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:50:28.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:28 smithi083 ceph-mon[29330]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:50:28.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:28 smithi094 ceph-mon[33032]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:50:30.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:30 smithi111 ceph-mon[32784]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:50:30.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:30 smithi083 ceph-mon[29330]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:50:30.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:30 smithi094 ceph-mon[33032]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:50:32.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:32 smithi111 ceph-mon[32784]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:50:32.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:32 smithi083 ceph-mon[29330]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:50:32.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:32 smithi094 ceph-mon[33032]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:50:34.320 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:34 smithi111 ceph-mon[32784]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T11:50:34.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:34 smithi083 ceph-mon[29330]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T11:50:34.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:34 smithi094 ceph-mon[33032]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T11:50:36.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:36 smithi111 ceph-mon[32784]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:36.323 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:36 smithi094 ceph-mon[33032]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:36.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:36 smithi083 ceph-mon[29330]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:38.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:38 smithi083 ceph-mon[29330]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:50:38.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:38 smithi094 ceph-mon[33032]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:50:38.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:38 smithi111 ceph-mon[32784]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:50:40.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:40 smithi083 ceph-mon[29330]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:40.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:40 smithi094 ceph-mon[33032]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:40.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:40 smithi111 ceph-mon[32784]: pgmap v214: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:42.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:42 smithi083 ceph-mon[29330]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:42.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:42 smithi094 ceph-mon[33032]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:42.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:42 smithi111 ceph-mon[32784]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:44.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:44 smithi083 ceph-mon[29330]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:50:44.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:44 smithi094 ceph-mon[33032]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:50:44.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:44 smithi111 ceph-mon[32784]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:50:46.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:46 smithi083 ceph-mon[29330]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:46.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:46 smithi094 ceph-mon[33032]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:46.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:46 smithi111 ceph-mon[32784]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:48.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:48 smithi083 ceph-mon[29330]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:50:48.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:48 smithi094 ceph-mon[33032]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:50:48.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:48 smithi111 ceph-mon[32784]: pgmap v218: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:50:50.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:50 smithi083 ceph-mon[29330]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:50.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:50 smithi094 ceph-mon[33032]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:50.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:50 smithi111 ceph-mon[32784]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:51.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:51 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:50:51.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:51 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:50:51.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:51 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:50:52.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:52 smithi083 ceph-mon[29330]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:52.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:52 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:50:52.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:52 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:50:52.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:52 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:50:52.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:52 smithi094 ceph-mon[33032]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:52.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:52 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:50:52.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:52 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:50:52.523 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:52 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:50:52.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:52 smithi111 ceph-mon[32784]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:52.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:52 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:50:52.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:52 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:50:52.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:52 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:50:54.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:54 smithi083 ceph-mon[29330]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:50:54.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:54 smithi094 ceph-mon[33032]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:50:54.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:54 smithi111 ceph-mon[32784]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:50:56.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:56 smithi083 ceph-mon[29330]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:56.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:56 smithi094 ceph-mon[33032]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:56.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:56 smithi111 ceph-mon[32784]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:50:58.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:50:58 smithi083 ceph-mon[29330]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:50:58.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:50:58 smithi094 ceph-mon[33032]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:50:58.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:50:58 smithi111 ceph-mon[32784]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:00.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:00 smithi083 ceph-mon[29330]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:00.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:00 smithi094 ceph-mon[33032]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:00.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:00 smithi111 ceph-mon[32784]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:02.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:02 smithi083 ceph-mon[29330]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:02.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:02 smithi094 ceph-mon[33032]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:02.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:02 smithi111 ceph-mon[32784]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:04.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:04 smithi083 ceph-mon[29330]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:51:04.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:04 smithi094 ceph-mon[33032]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:51:04.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:04 smithi111 ceph-mon[32784]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:51:06.371 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:06 smithi111 ceph-mon[32784]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:51:06.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:06 smithi083 ceph-mon[29330]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:51:06.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:06 smithi094 ceph-mon[33032]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:51:08.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:08 smithi083 ceph-mon[29330]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:51:08.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:08 smithi094 ceph-mon[33032]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:51:08.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:08 smithi111 ceph-mon[32784]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:51:10.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:10 smithi083 ceph-mon[29330]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:51:10.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:10 smithi094 ceph-mon[33032]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:51:10.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:10 smithi111 ceph-mon[32784]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:51:11.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:11 smithi083 ceph-mon[29330]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T11:51:11.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:11 smithi094 ceph-mon[33032]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T11:51:11.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:11 smithi111 ceph-mon[32784]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T11:51:12.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:12 smithi083 ceph-mon[29330]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T11:51:12.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:12 smithi094 ceph-mon[33032]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T11:51:12.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:12 smithi111 ceph-mon[32784]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T11:51:14.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:14 smithi083 ceph-mon[29330]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:51:14.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:14 smithi094 ceph-mon[33032]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:51:14.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:14 smithi111 ceph-mon[32784]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:51:16.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:16 smithi083 ceph-mon[29330]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:51:16.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:16 smithi094 ceph-mon[33032]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:51:16.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:16 smithi111 ceph-mon[32784]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:51:18.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:17 smithi094 ceph-mon[33032]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:51:18.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:17 smithi094 ceph-mon[33032]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T11:51:18.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:17 smithi111 ceph-mon[32784]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:51:18.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:17 smithi111 ceph-mon[32784]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T11:51:18.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:17 smithi083 ceph-mon[29330]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:51:18.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:17 smithi083 ceph-mon[29330]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T11:51:20.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:19 smithi094 ceph-mon[33032]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 249 B/s wr, 12 op/s 2024-04-25T11:51:20.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:19 smithi111 ceph-mon[32784]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 249 B/s wr, 12 op/s 2024-04-25T11:51:20.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:19 smithi083 ceph-mon[29330]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 249 B/s wr, 12 op/s 2024-04-25T11:51:22.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:21 smithi094 ceph-mon[33032]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:51:22.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:21 smithi111 ceph-mon[32784]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:51:22.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:21 smithi083 ceph-mon[29330]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:51:24.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:23 smithi094 ceph-mon[33032]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:51:24.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:23 smithi111 ceph-mon[32784]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:51:24.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:23 smithi083 ceph-mon[29330]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:51:25.956 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:25 smithi083 ceph-mon[29330]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:51:26.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:25 smithi094 ceph-mon[33032]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:51:26.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:25 smithi111 ceph-mon[32784]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:51:28.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:27 smithi094 ceph-mon[33032]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:51:28.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:27 smithi111 ceph-mon[32784]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:51:28.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:27 smithi083 ceph-mon[29330]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:51:30.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:29 smithi094 ceph-mon[33032]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T11:51:30.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:29 smithi111 ceph-mon[32784]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T11:51:30.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:29 smithi083 ceph-mon[29330]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T11:51:32.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:31 smithi094 ceph-mon[33032]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:32.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:31 smithi111 ceph-mon[32784]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:32.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:31 smithi083 ceph-mon[29330]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:34.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:33 smithi094 ceph-mon[33032]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:34.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:33 smithi111 ceph-mon[32784]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:34.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:33 smithi083 ceph-mon[29330]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:36.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:35 smithi094 ceph-mon[33032]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:36.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:35 smithi111 ceph-mon[32784]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:36.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:35 smithi083 ceph-mon[29330]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:38.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:37 smithi094 ceph-mon[33032]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:38.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:37 smithi111 ceph-mon[32784]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:38.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:37 smithi083 ceph-mon[29330]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:40.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:39 smithi094 ceph-mon[33032]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:40.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:39 smithi111 ceph-mon[32784]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:40.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:39 smithi083 ceph-mon[29330]: pgmap v246: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:42.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:41 smithi094 ceph-mon[33032]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:42.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:41 smithi111 ceph-mon[32784]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:42.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:41 smithi083 ceph-mon[29330]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:44.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:43 smithi094 ceph-mon[33032]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:44.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:43 smithi111 ceph-mon[32784]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:44.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:43 smithi083 ceph-mon[29330]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:46.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:45 smithi094 ceph-mon[33032]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:46.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:45 smithi111 ceph-mon[32784]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:46.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:45 smithi083 ceph-mon[29330]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:48.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:47 smithi094 ceph-mon[33032]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:48.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:47 smithi111 ceph-mon[32784]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:48.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:47 smithi083 ceph-mon[29330]: pgmap v250: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:50.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:49 smithi094 ceph-mon[33032]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:50.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:49 smithi111 ceph-mon[32784]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:50.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:49 smithi083 ceph-mon[29330]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:52.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:51 smithi094 ceph-mon[33032]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:52.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:51 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:51:52.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:51 smithi111 ceph-mon[32784]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:52.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:51 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:51:52.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:51 smithi083 ceph-mon[29330]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:52.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:51 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:51:53.020 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:52 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:51:53.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:52 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:51:53.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:52 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:51:53.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:52 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:51:53.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:52 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:51:53.069 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:52 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:51:53.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:52 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:51:53.153 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:52 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:51:53.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:52 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:51:54.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:54 smithi111 ceph-mon[32784]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:54.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:54 smithi083 ceph-mon[29330]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:54.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:54 smithi094 ceph-mon[33032]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:56.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:56 smithi111 ceph-mon[32784]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:56.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:56 smithi083 ceph-mon[29330]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:56.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:56 smithi094 ceph-mon[33032]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:51:58.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:51:58 smithi111 ceph-mon[32784]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:58.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:51:58 smithi083 ceph-mon[29330]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:51:58.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:51:58 smithi094 ceph-mon[33032]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:00.319 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:00 smithi111 ceph-mon[32784]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:00.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:00 smithi083 ceph-mon[29330]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:00.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:00 smithi094 ceph-mon[33032]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:02.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:02 smithi083 ceph-mon[29330]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:02.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:02 smithi094 ceph-mon[33032]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:02.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:02 smithi111 ceph-mon[32784]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:04.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:04 smithi083 ceph-mon[29330]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:52:04.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:04 smithi094 ceph-mon[33032]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:52:04.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:04 smithi111 ceph-mon[32784]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:52:06.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:06 smithi083 ceph-mon[29330]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:52:06.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:06 smithi094 ceph-mon[33032]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:52:06.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:06 smithi111 ceph-mon[32784]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:52:08.372 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:08 smithi111 ceph-mon[32784]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:52:08.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:08 smithi083 ceph-mon[29330]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:52:08.520 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:08 smithi094 ceph-mon[33032]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:52:10.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:10 smithi083 ceph-mon[29330]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:52:10.522 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:10 smithi094 ceph-mon[33032]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:52:10.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:10 smithi111 ceph-mon[32784]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:52:11.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:11 smithi083 ceph-mon[29330]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T11:52:11.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:11 smithi094 ceph-mon[33032]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T11:52:11.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:11 smithi111 ceph-mon[32784]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T11:52:12.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:12 smithi083 ceph-mon[29330]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T11:52:12.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:12 smithi094 ceph-mon[33032]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T11:52:12.569 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:12 smithi111 ceph-mon[32784]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-25T11:52:14.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:14 smithi083 ceph-mon[29330]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:52:14.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:14 smithi094 ceph-mon[33032]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:52:14.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:14 smithi111 ceph-mon[32784]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:52:16.404 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:16 smithi083 ceph-mon[29330]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:52:16.521 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:16 smithi094 ceph-mon[33032]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:52:16.570 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:16 smithi111 ceph-mon[32784]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:52:18.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:17 smithi094 ceph-mon[33032]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:52:18.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:17 smithi094 ceph-mon[33032]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T11:52:18.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:17 smithi111 ceph-mon[32784]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:52:18.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:17 smithi111 ceph-mon[32784]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T11:52:18.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:17 smithi083 ceph-mon[29330]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:52:18.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:17 smithi083 ceph-mon[29330]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T11:52:20.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:19 smithi094 ceph-mon[33032]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 248 B/s wr, 12 op/s 2024-04-25T11:52:20.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:19 smithi111 ceph-mon[32784]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 248 B/s wr, 12 op/s 2024-04-25T11:52:20.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:19 smithi083 ceph-mon[29330]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 248 B/s wr, 12 op/s 2024-04-25T11:52:22.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:21 smithi094 ceph-mon[33032]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:52:22.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:21 smithi111 ceph-mon[32784]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:52:22.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:21 smithi083 ceph-mon[29330]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:52:24.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:23 smithi094 ceph-mon[33032]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:52:24.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:23 smithi111 ceph-mon[32784]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:52:24.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:23 smithi083 ceph-mon[29330]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:52:26.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:25 smithi094 ceph-mon[33032]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:52:26.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:25 smithi111 ceph-mon[32784]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:52:26.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:25 smithi083 ceph-mon[29330]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:52:28.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:27 smithi094 ceph-mon[33032]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:52:28.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:27 smithi111 ceph-mon[32784]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:52:28.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:27 smithi083 ceph-mon[29330]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:52:30.023 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:29 smithi094 ceph-mon[33032]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T11:52:30.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:29 smithi111 ceph-mon[32784]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T11:52:30.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:29 smithi083 ceph-mon[29330]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-25T11:52:32.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:31 smithi094 ceph-mon[33032]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:32.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:31 smithi111 ceph-mon[32784]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:32.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:31 smithi083 ceph-mon[29330]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:34.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:33 smithi094 ceph-mon[33032]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:34.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:33 smithi111 ceph-mon[32784]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:34.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:33 smithi083 ceph-mon[29330]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:36.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:35 smithi094 ceph-mon[33032]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:36.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:35 smithi111 ceph-mon[32784]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:36.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:35 smithi083 ceph-mon[29330]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:38.021 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:37 smithi094 ceph-mon[33032]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:38.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:37 smithi111 ceph-mon[32784]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:38.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:37 smithi083 ceph-mon[29330]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:40.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:39 smithi094 ceph-mon[33032]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:40.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:39 smithi111 ceph-mon[32784]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:40.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:39 smithi083 ceph-mon[29330]: pgmap v278: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:42.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:41 smithi094 ceph-mon[33032]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:42.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:41 smithi111 ceph-mon[32784]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:42.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:41 smithi083 ceph-mon[29330]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:44.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:43 smithi094 ceph-mon[33032]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:44.070 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:43 smithi111 ceph-mon[32784]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:44.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:43 smithi083 ceph-mon[29330]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:46.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:45 smithi094 ceph-mon[33032]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:46.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:45 smithi111 ceph-mon[32784]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:46.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:45 smithi083 ceph-mon[29330]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:48.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:47 smithi094 ceph-mon[33032]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:48.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:47 smithi111 ceph-mon[32784]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:48.154 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:47 smithi083 ceph-mon[29330]: pgmap v282: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:50.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:49 smithi094 ceph-mon[33032]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:50.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:49 smithi111 ceph-mon[32784]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:50.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:49 smithi083 ceph-mon[29330]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:52.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:51 smithi094 ceph-mon[33032]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:52.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:51 smithi111 ceph-mon[32784]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:52.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:51 smithi083 ceph-mon[29330]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:53.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:52 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:52:53.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:52 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:52:53.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:52 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:52:54.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:53 smithi094 ceph-mon[33032]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:54.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:53 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:52:54.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:53 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:52:54.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:53 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:52:54.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:53 smithi111 ceph-mon[32784]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:54.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:53 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:52:54.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:53 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:52:54.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:53 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:52:54.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:53 smithi083 ceph-mon[29330]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:54.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:53 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:52:54.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:53 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:52:54.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:53 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:52:56.022 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:55 smithi094 ceph-mon[33032]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:56.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:55 smithi111 ceph-mon[32784]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:56.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:55 smithi083 ceph-mon[29330]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:52:58.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:57 smithi111 ceph-mon[32784]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:58.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:57 smithi083 ceph-mon[29330]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:52:58.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:57 smithi094 ceph-mon[33032]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:53:00.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:52:59 smithi111 ceph-mon[32784]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:00.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:52:59 smithi083 ceph-mon[29330]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:00.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:52:59 smithi094 ceph-mon[33032]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:02.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:01 smithi111 ceph-mon[32784]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:02.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:01 smithi083 ceph-mon[29330]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:02.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:01 smithi094 ceph-mon[33032]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:04.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:03 smithi111 ceph-mon[32784]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:53:04.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:03 smithi083 ceph-mon[29330]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:53:04.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:03 smithi094 ceph-mon[33032]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:53:06.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:05 smithi111 ceph-mon[32784]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:53:06.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:05 smithi083 ceph-mon[29330]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:53:06.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:05 smithi094 ceph-mon[33032]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:53:08.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:07 smithi111 ceph-mon[32784]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:53:08.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:07 smithi083 ceph-mon[29330]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:53:08.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:07 smithi094 ceph-mon[33032]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:53:10.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:09 smithi111 ceph-mon[32784]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:53:10.075 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:09 smithi094 ceph-mon[33032]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:53:10.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:09 smithi083 ceph-mon[29330]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:53:12.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:11 smithi111 ceph-mon[32784]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:53:12.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:11 smithi111 ceph-mon[32784]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T11:53:12.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:11 smithi083 ceph-mon[29330]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:53:12.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:11 smithi083 ceph-mon[29330]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T11:53:12.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:11 smithi094 ceph-mon[33032]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:53:12.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:11 smithi094 ceph-mon[33032]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T11:53:14.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:13 smithi111 ceph-mon[32784]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:53:14.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:13 smithi083 ceph-mon[29330]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:53:14.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:13 smithi094 ceph-mon[33032]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:53:16.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:15 smithi111 ceph-mon[32784]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:53:16.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:15 smithi083 ceph-mon[29330]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:53:16.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:15 smithi094 ceph-mon[33032]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:53:18.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:17 smithi111 ceph-mon[32784]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:53:18.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:17 smithi083 ceph-mon[29330]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:53:18.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:17 smithi094 ceph-mon[33032]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:53:20.071 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:19 smithi111 ceph-mon[32784]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:53:20.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:19 smithi083 ceph-mon[29330]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:53:20.272 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:19 smithi094 ceph-mon[33032]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:53:22.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:21 smithi111 ceph-mon[32784]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:53:22.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:21 smithi111 ceph-mon[32784]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T11:53:22.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:21 smithi083 ceph-mon[29330]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:53:22.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:21 smithi083 ceph-mon[29330]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T11:53:22.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:21 smithi094 ceph-mon[33032]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:53:22.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:21 smithi094 ceph-mon[33032]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T11:53:24.155 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:23 smithi083 ceph-mon[29330]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:53:24.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:23 smithi094 ceph-mon[33032]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:53:24.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:23 smithi111 ceph-mon[32784]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:53:26.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:25 smithi083 ceph-mon[29330]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:53:26.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:25 smithi094 ceph-mon[33032]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:53:26.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:25 smithi111 ceph-mon[32784]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:53:28.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:27 smithi083 ceph-mon[29330]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:53:28.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:27 smithi094 ceph-mon[33032]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:53:28.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:27 smithi111 ceph-mon[32784]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:53:30.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:29 smithi083 ceph-mon[29330]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:53:30.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:29 smithi094 ceph-mon[33032]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:53:30.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:29 smithi111 ceph-mon[32784]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:53:32.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:31 smithi083 ceph-mon[29330]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:53:32.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:31 smithi094 ceph-mon[33032]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:53:32.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:31 smithi111 ceph-mon[32784]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-25T11:53:34.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:33 smithi083 ceph-mon[29330]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T11:53:34.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:33 smithi094 ceph-mon[33032]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T11:53:34.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:33 smithi111 ceph-mon[32784]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-25T11:53:36.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:35 smithi083 ceph-mon[29330]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:36.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:35 smithi094 ceph-mon[33032]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:36.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:35 smithi111 ceph-mon[32784]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:38.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:37 smithi083 ceph-mon[29330]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:53:38.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:37 smithi094 ceph-mon[33032]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:53:38.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:37 smithi111 ceph-mon[32784]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:53:40.124 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:39 smithi111 ceph-mon[32784]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:40.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:39 smithi083 ceph-mon[29330]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:40.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:39 smithi094 ceph-mon[33032]: pgmap v310: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:42.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:41 smithi083 ceph-mon[29330]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:42.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:41 smithi094 ceph-mon[33032]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:42.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:41 smithi111 ceph-mon[32784]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:42.913 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T11:53:43.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.037+0000 7f0782e10640 1 -- 172.21.15.83:0/2404063004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c101a90 msgr2=0x7f077c105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.037+0000 7f0782e10640 1 --2- 172.21.15.83:0/2404063004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c101a90 0x7f077c105ad0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0768009900 tx=0x7f076802a410 comp rx=0 tx=0).stop 2024-04-25T11:53:43.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.037+0000 7f0782e10640 1 -- 172.21.15.83:0/2404063004 shutdown_connections 2024-04-25T11:53:43.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.037+0000 7f0782e10640 1 --2- 172.21.15.83:0/2404063004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c101a90 0x7f077c105ad0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.037+0000 7f0782e10640 1 --2- 172.21.15.83:0/2404063004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077c1061b0 0x7f077c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.037+0000 7f0782e10640 1 --2- 172.21.15.83:0/2404063004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f077c1010e0 0x7f077c1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.037+0000 7f0782e10640 1 -- 172.21.15.83:0/2404063004 >> 172.21.15.83:0/2404063004 conn(0x7f077c0771d0 msgr2=0x7f077c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:53:43.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.037+0000 7f0782e10640 1 -- 172.21.15.83:0/2404063004 shutdown_connections 2024-04-25T11:53:43.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.037+0000 7f0782e10640 1 -- 172.21.15.83:0/2404063004 wait complete. 2024-04-25T11:53:43.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f0782e10640 1 Processor -- start 2024-04-25T11:53:43.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f0782e10640 1 -- start start 2024-04-25T11:53:43.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f0782e10640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f077c1010e0 0x7f077c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:43.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f0782e10640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c1061b0 0x7f077c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f0782e10640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077c19cfd0 0x7f077c19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f0782e10640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f077c111150 con 0x7f077c1061b0 2024-04-25T11:53:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f0782e10640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f077c110fd0 con 0x7f077c1010e0 2024-04-25T11:53:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f0782e10640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f077c1112d0 con 0x7f077c19cfd0 2024-04-25T11:53:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f077bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c1061b0 0x7f077c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f077bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c1061b0 0x7f077c198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45594/0 (socket says 172.21.15.83:45594) 2024-04-25T11:53:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f077bfff640 1 -- 172.21.15.83:0/2439766795 learned_addr learned my addr 172.21.15.83:0/2439766795 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:53:43.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f07813ad640 1 --2- 172.21.15.83:0/2439766795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077c19cfd0 0x7f077c19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f0780bac640 1 --2- 172.21.15.83:0/2439766795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f077c1010e0 0x7f077c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f077bfff640 1 -- 172.21.15.83:0/2439766795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077c19cfd0 msgr2=0x7f077c19d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f077bfff640 1 --2- 172.21.15.83:0/2439766795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077c19cfd0 0x7f077c19d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f077bfff640 1 -- 172.21.15.83:0/2439766795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f077c1010e0 msgr2=0x7f077c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f077bfff640 1 --2- 172.21.15.83:0/2439766795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f077c1010e0 0x7f077c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.038+0000 7f077bfff640 1 -- 172.21.15.83:0/2439766795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077c19dac0 con 0x7f077c1061b0 2024-04-25T11:53:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.039+0000 7f0780bac640 1 --2- 172.21.15.83:0/2439766795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f077c1010e0 0x7f077c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:53:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.039+0000 7f077bfff640 1 --2- 172.21.15.83:0/2439766795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c1061b0 0x7f077c198c50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0768032fd0 tx=0x7f0768004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:53:43.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.039+0000 7f0779ffb640 1 -- 172.21.15.83:0/2439766795 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076803f030 con 0x7f077c1061b0 2024-04-25T11:53:43.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.039+0000 7f0779ffb640 1 -- 172.21.15.83:0/2439766795 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0768004490 con 0x7f077c1061b0 2024-04-25T11:53:43.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.039+0000 7f0782e10640 1 -- 172.21.15.83:0/2439766795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077c19dd50 con 0x7f077c1061b0 2024-04-25T11:53:43.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.039+0000 7f0779ffb640 1 -- 172.21.15.83:0/2439766795 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768035070 con 0x7f077c1061b0 2024-04-25T11:53:43.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.039+0000 7f0782e10640 1 -- 172.21.15.83:0/2439766795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f077c1a5570 con 0x7f077c1061b0 2024-04-25T11:53:43.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.040+0000 7f0782e10640 1 -- 172.21.15.83:0/2439766795 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f077c04fce0 con 0x7f077c1061b0 2024-04-25T11:53:43.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.041+0000 7f0779ffb640 1 -- 172.21.15.83:0/2439766795 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0768042030 con 0x7f077c1061b0 2024-04-25T11:53:43.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.042+0000 7f0779ffb640 1 --2- 172.21.15.83:0/2439766795 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f076c0a3e40 0x7f076c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:43.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.042+0000 7f0779ffb640 1 -- 172.21.15.83:0/2439766795 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f07680f0d00 con 0x7f077c1061b0 2024-04-25T11:53:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.042+0000 7f0780bac640 1 --2- 172.21.15.83:0/2439766795 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f076c0a3e40 0x7f076c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.042+0000 7f0780bac640 1 --2- 172.21.15.83:0/2439766795 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f076c0a3e40 0x7f076c0a6300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f07700058c0 tx=0x7f0770005850 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:53:43.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.047+0000 7f0779ffb640 1 -- 172.21.15.83:0/2439766795 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07680bd3a0 con 0x7f077c1061b0 2024-04-25T11:53:43.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.246+0000 7f0782e10640 1 -- 172.21.15.83:0/2439766795 --> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f077c101a90 con 0x7f076c0a3e40 2024-04-25T11:53:43.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.255+0000 7f0779ffb640 1 -- 172.21.15.83:0/2439766795 <== mgr.15360 v2:172.21.15.83:6800/4112952368 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f077c101a90 con 0x7f076c0a3e40 2024-04-25T11:53:43.243 INFO:teuthology.orchestra.run.smithi083.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T11:43:41", "first": "2024-04-25T11:44:00", "last": "2024-04-25T11:53:00", "last_pruned": "2024-04-25T11:53:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T11:53:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.257+0000 7f0782e10640 1 -- 172.21.15.83:0/2439766795 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f076c0a3e40 msgr2=0x7f076c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.257+0000 7f0782e10640 1 --2- 172.21.15.83:0/2439766795 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f076c0a3e40 0x7f076c0a6300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f07700058c0 tx=0x7f0770005850 comp rx=0 tx=0).stop 2024-04-25T11:53:43.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.257+0000 7f0782e10640 1 -- 172.21.15.83:0/2439766795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c1061b0 msgr2=0x7f077c198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.257+0000 7f0782e10640 1 --2- 172.21.15.83:0/2439766795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c1061b0 0x7f077c198c50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0768032fd0 tx=0x7f0768004270 comp rx=0 tx=0).stop 2024-04-25T11:53:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.257+0000 7f0782e10640 1 -- 172.21.15.83:0/2439766795 shutdown_connections 2024-04-25T11:53:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.257+0000 7f0782e10640 1 --2- 172.21.15.83:0/2439766795 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f076c0a3e40 0x7f076c0a6300 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.257+0000 7f0782e10640 1 --2- 172.21.15.83:0/2439766795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f077c1061b0 0x7f077c198c50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.257+0000 7f0782e10640 1 --2- 172.21.15.83:0/2439766795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077c19cfd0 0x7f077c19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.257+0000 7f0782e10640 1 --2- 172.21.15.83:0/2439766795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f077c1010e0 0x7f077c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.257+0000 7f0782e10640 1 -- 172.21.15.83:0/2439766795 >> 172.21.15.83:0/2439766795 conn(0x7f077c0771d0 msgr2=0x7f077c06aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:53:43.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.257+0000 7f0782e10640 1 -- 172.21.15.83:0/2439766795 shutdown_connections 2024-04-25T11:53:43.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.257+0000 7f0782e10640 1 -- 172.21.15.83:0/2439766795 wait complete. 2024-04-25T11:53:43.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T11:53:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.367+0000 7f660b26b640 1 -- 172.21.15.83:0/1475183326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660410e930 msgr2=0x7f6604110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.367+0000 7f660b26b640 1 --2- 172.21.15.83:0/1475183326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660410e930 0x7f6604110d20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009900 tx=0x7f65fc02a410 comp rx=0 tx=0).stop 2024-04-25T11:53:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.367+0000 7f660b26b640 1 -- 172.21.15.83:0/1475183326 shutdown_connections 2024-04-25T11:53:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.367+0000 7f660b26b640 1 --2- 172.21.15.83:0/1475183326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f660410e930 0x7f6604110d20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.367+0000 7f660b26b640 1 --2- 172.21.15.83:0/1475183326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66041016a0 0x7f660410e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.367+0000 7f660b26b640 1 --2- 172.21.15.83:0/1475183326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6604100d80 0x7f6604101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.367+0000 7f660b26b640 1 -- 172.21.15.83:0/1475183326 >> 172.21.15.83:0/1475183326 conn(0x7f6604077fd0 msgr2=0x7f66040783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:53:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.367+0000 7f660b26b640 1 -- 172.21.15.83:0/1475183326 shutdown_connections 2024-04-25T11:53:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.368+0000 7f660b26b640 1 -- 172.21.15.83:0/1475183326 wait complete. 2024-04-25T11:53:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.368+0000 7f660b26b640 1 Processor -- start 2024-04-25T11:53:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.368+0000 7f660b26b640 1 -- start start 2024-04-25T11:53:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.368+0000 7f660b26b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6604100d80 0x7f66041a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.368+0000 7f660b26b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66041016a0 0x7f66041a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.368+0000 7f660b26b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f660410e930 0x7f660419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.368+0000 7f660b26b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66041143b0 con 0x7f66041016a0 2024-04-25T11:53:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.368+0000 7f660b26b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6604114230 con 0x7f6604100d80 2024-04-25T11:53:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.368+0000 7f660b26b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6604114530 con 0x7f660410e930 2024-04-25T11:53:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.368+0000 7f6608806640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66041016a0 0x7f66041a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.368+0000 7f6608806640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66041016a0 0x7f66041a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45620/0 (socket says 172.21.15.83:45620) 2024-04-25T11:53:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.368+0000 7f6608806640 1 -- 172.21.15.83:0/154951191 learned_addr learned my addr 172.21.15.83:0/154951191 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:53:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.368+0000 7f6609007640 1 --2- 172.21.15.83:0/154951191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6604100d80 0x7f66041a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.369+0000 7f6608806640 1 -- 172.21.15.83:0/154951191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f660410e930 msgr2=0x7f660419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.369+0000 7f6608806640 1 --2- 172.21.15.83:0/154951191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f660410e930 0x7f660419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.369+0000 7f6608806640 1 -- 172.21.15.83:0/154951191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6604100d80 msgr2=0x7f66041a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.369+0000 7f6608806640 1 --2- 172.21.15.83:0/154951191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6604100d80 0x7f66041a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.369+0000 7f6608806640 1 -- 172.21.15.83:0/154951191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660419aff0 con 0x7f66041016a0 2024-04-25T11:53:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.369+0000 7f6608806640 1 --2- 172.21.15.83:0/154951191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66041016a0 0x7f66041a0bb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f65f800c970 tx=0x7f65f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:53:43.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.369+0000 7f66027fc640 1 -- 172.21.15.83:0/154951191 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f8007bb0 con 0x7f66041016a0 2024-04-25T11:53:43.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.369+0000 7f66027fc640 1 -- 172.21.15.83:0/154951191 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65f8007d50 con 0x7f66041016a0 2024-04-25T11:53:43.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.369+0000 7f660b26b640 1 -- 172.21.15.83:0/154951191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660419b2e0 con 0x7f66041016a0 2024-04-25T11:53:43.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.369+0000 7f66027fc640 1 -- 172.21.15.83:0/154951191 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f800f450 con 0x7f66041016a0 2024-04-25T11:53:43.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.369+0000 7f660b26b640 1 -- 172.21.15.83:0/154951191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66041a75f0 con 0x7f66041016a0 2024-04-25T11:53:43.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.370+0000 7f660b26b640 1 -- 172.21.15.83:0/154951191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6604069de0 con 0x7f66041016a0 2024-04-25T11:53:43.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.371+0000 7f66027fc640 1 -- 172.21.15.83:0/154951191 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f65f8004f00 con 0x7f66041016a0 2024-04-25T11:53:43.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.371+0000 7f66027fc640 1 --2- 172.21.15.83:0/154951191 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f65ec0a3dc0 0x7f65ec0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:43.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.372+0000 7f6609007640 1 --2- 172.21.15.83:0/154951191 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f65ec0a3dc0 0x7f65ec0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:43.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.372+0000 7f66027fc640 1 -- 172.21.15.83:0/154951191 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f65f80d0870 con 0x7f66041016a0 2024-04-25T11:53:43.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.372+0000 7f6609007640 1 --2- 172.21.15.83:0/154951191 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f65ec0a3dc0 0x7f65ec0a6280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f65f4005fd0 tx=0x7f65f4009680 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:53:43.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.375+0000 7f66027fc640 1 -- 172.21.15.83:0/154951191 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65f8014030 con 0x7f66041016a0 2024-04-25T11:53:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.573+0000 7f660b26b640 1 -- 172.21.15.83:0/154951191 --> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f660419b510 con 0x7f65ec0a3dc0 2024-04-25T11:53:43.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.580+0000 7f66027fc640 1 -- 172.21.15.83:0/154951191 <== mgr.15360 v2:172.21.15.83:6800/4112952368 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f660419b510 con 0x7f65ec0a3dc0 2024-04-25T11:53:43.568 INFO:teuthology.orchestra.run.smithi083.stdout:/ 1m 6m3h 2024-04-25T11:53:43.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.582+0000 7f660b26b640 1 -- 172.21.15.83:0/154951191 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f65ec0a3dc0 msgr2=0x7f65ec0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.582+0000 7f660b26b640 1 --2- 172.21.15.83:0/154951191 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f65ec0a3dc0 0x7f65ec0a6280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f65f4005fd0 tx=0x7f65f4009680 comp rx=0 tx=0).stop 2024-04-25T11:53:43.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.582+0000 7f660b26b640 1 -- 172.21.15.83:0/154951191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66041016a0 msgr2=0x7f66041a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.582+0000 7f660b26b640 1 --2- 172.21.15.83:0/154951191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66041016a0 0x7f66041a0bb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f65f800c970 tx=0x7f65f800ce30 comp rx=0 tx=0).stop 2024-04-25T11:53:43.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.582+0000 7f660b26b640 1 -- 172.21.15.83:0/154951191 shutdown_connections 2024-04-25T11:53:43.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.582+0000 7f660b26b640 1 --2- 172.21.15.83:0/154951191 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f65ec0a3dc0 0x7f65ec0a6280 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.582+0000 7f660b26b640 1 --2- 172.21.15.83:0/154951191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66041016a0 0x7f66041a0bb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.582+0000 7f660b26b640 1 --2- 172.21.15.83:0/154951191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f660410e930 0x7f660419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.582+0000 7f660b26b640 1 --2- 172.21.15.83:0/154951191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6604100d80 0x7f66041a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.582+0000 7f660b26b640 1 -- 172.21.15.83:0/154951191 >> 172.21.15.83:0/154951191 conn(0x7f6604077fd0 msgr2=0x7f660410fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:53:43.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.582+0000 7f660b26b640 1 -- 172.21.15.83:0/154951191 shutdown_connections 2024-04-25T11:53:43.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.582+0000 7f660b26b640 1 -- 172.21.15.83:0/154951191 wait complete. 2024-04-25T11:53:43.580 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T11:53:43.592 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:53:43.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.703+0000 7ff406ccf640 1 -- 172.21.15.83:0/1629809457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4001010a0 msgr2=0x7ff400101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.703+0000 7ff406ccf640 1 --2- 172.21.15.83:0/1629809457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4001010a0 0x7ff400101480 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009ca0 tx=0x7ff3f802a8c0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.703+0000 7ff406ccf640 1 -- 172.21.15.83:0/1629809457 shutdown_connections 2024-04-25T11:53:43.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.703+0000 7ff406ccf640 1 --2- 172.21.15.83:0/1629809457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff400106170 0x7ff40010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.703+0000 7ff406ccf640 1 --2- 172.21.15.83:0/1629809457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff400101a50 0x7ff400105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.703+0000 7ff406ccf640 1 --2- 172.21.15.83:0/1629809457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4001010a0 0x7ff400101480 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.703+0000 7ff406ccf640 1 -- 172.21.15.83:0/1629809457 >> 172.21.15.83:0/1629809457 conn(0x7ff400077f50 msgr2=0x7ff400078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:53:43.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.703+0000 7ff406ccf640 1 -- 172.21.15.83:0/1629809457 shutdown_connections 2024-04-25T11:53:43.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.703+0000 7ff406ccf640 1 -- 172.21.15.83:0/1629809457 wait complete. 2024-04-25T11:53:43.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.704+0000 7ff406ccf640 1 Processor -- start 2024-04-25T11:53:43.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.704+0000 7ff406ccf640 1 -- start start 2024-04-25T11:53:43.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.704+0000 7ff406ccf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4001010a0 0x7ff400198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:43.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.704+0000 7ff406ccf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff400106170 0x7ff400198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:43.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.704+0000 7ff406ccf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40019cf30 0x7ff40019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:43.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.704+0000 7ff406ccf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4001110b0 con 0x7ff4001010a0 2024-04-25T11:53:43.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.704+0000 7ff406ccf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff400110f30 con 0x7ff40019cf30 2024-04-25T11:53:43.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.704+0000 7ff406ccf640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff400111230 con 0x7ff400106170 2024-04-25T11:53:43.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.704+0000 7ff404a6b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4001010a0 0x7ff400198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:43.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.704+0000 7ff404a6b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4001010a0 0x7ff400198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45634/0 (socket says 172.21.15.83:45634) 2024-04-25T11:53:43.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.704+0000 7ff404a6b640 1 -- 172.21.15.83:0/3304799787 learned_addr learned my addr 172.21.15.83:0/3304799787 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:53:43.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.704+0000 7ff3fffff640 1 --2- 172.21.15.83:0/3304799787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff400106170 0x7ff400198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:43.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.704+0000 7ff40526c640 1 --2- 172.21.15.83:0/3304799787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40019cf30 0x7ff40019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:43.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.705+0000 7ff404a6b640 1 -- 172.21.15.83:0/3304799787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff400106170 msgr2=0x7ff400198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.705+0000 7ff404a6b640 1 --2- 172.21.15.83:0/3304799787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff400106170 0x7ff400198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.705+0000 7ff404a6b640 1 -- 172.21.15.83:0/3304799787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40019cf30 msgr2=0x7ff40019d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.705+0000 7ff404a6b640 1 --2- 172.21.15.83:0/3304799787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40019cf30 0x7ff40019d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.705+0000 7ff404a6b640 1 -- 172.21.15.83:0/3304799787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40019da20 con 0x7ff4001010a0 2024-04-25T11:53:43.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.705+0000 7ff3fffff640 1 --2- 172.21.15.83:0/3304799787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff400106170 0x7ff400198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:53:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.705+0000 7ff404a6b640 1 --2- 172.21.15.83:0/3304799787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4001010a0 0x7ff400198670 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff3f802ade0 tx=0x7ff3f8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:53:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.705+0000 7ff3fdffb640 1 -- 172.21.15.83:0/3304799787 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8003d60 con 0x7ff4001010a0 2024-04-25T11:53:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.705+0000 7ff3fdffb640 1 -- 172.21.15.83:0/3304799787 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3f80044f0 con 0x7ff4001010a0 2024-04-25T11:53:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.705+0000 7ff406ccf640 1 -- 172.21.15.83:0/3304799787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40019dcb0 con 0x7ff4001010a0 2024-04-25T11:53:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.705+0000 7ff3fdffb640 1 -- 172.21.15.83:0/3304799787 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f802fd10 con 0x7ff4001010a0 2024-04-25T11:53:43.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.705+0000 7ff406ccf640 1 -- 172.21.15.83:0/3304799787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4001a54d0 con 0x7ff4001010a0 2024-04-25T11:53:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.706+0000 7ff406ccf640 1 -- 172.21.15.83:0/3304799787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff400068d70 con 0x7ff4001010a0 2024-04-25T11:53:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.707+0000 7ff3fdffb640 1 -- 172.21.15.83:0/3304799787 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff3f80046a0 con 0x7ff4001010a0 2024-04-25T11:53:43.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.708+0000 7ff3fdffb640 1 --2- 172.21.15.83:0/3304799787 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7ff3ec0a3d20 0x7ff3ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:43.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.708+0000 7ff3fffff640 1 --2- 172.21.15.83:0/3304799787 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7ff3ec0a3d20 0x7ff3ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:43.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.708+0000 7ff3fdffb640 1 -- 172.21.15.83:0/3304799787 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7ff3f80f2000 con 0x7ff4001010a0 2024-04-25T11:53:43.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.708+0000 7ff3fffff640 1 --2- 172.21.15.83:0/3304799787 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7ff3ec0a3d20 0x7ff3ec0a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff400100940 tx=0x7ff3e800a5c0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:53:43.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.712+0000 7ff3fdffb640 1 -- 172.21.15.83:0/3304799787 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff3f80be7b0 con 0x7ff4001010a0 2024-04-25T11:53:43.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.891+0000 7ff406ccf640 1 -- 172.21.15.83:0/3304799787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4001999c0 con 0x7ff4001010a0 2024-04-25T11:53:43.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.893+0000 7ff3fdffb640 1 -- 172.21.15.83:0/3304799787 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+23048 (secure 0 0 0) 0x7ff3f80be0b0 con 0x7ff4001010a0 2024-04-25T11:53:43.881 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:53:43.881 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":97,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:53:21.730028+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:41:42.546991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:41:43.764226+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/1157946519":"2024-04-26T11:43:40.200248+0000","172.21.15.83:0/3934887058":"2024-04-26T11:43:40.200248+0000","172.21.15.83:0/457699907":"2024-04-26T11:43:40.200248+0000","172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1837678414":"2024-04-26T11:43:40.200248+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6800/1157946519":"2024-04-26T11:43:40.200248+0000","172.21.15.83:0/3642996010":"2024-04-26T11:43:40.200248+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:53:43.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.895+0000 7ff406ccf640 1 -- 172.21.15.83:0/3304799787 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7ff3ec0a3d20 msgr2=0x7ff3ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.895+0000 7ff406ccf640 1 --2- 172.21.15.83:0/3304799787 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7ff3ec0a3d20 0x7ff3ec0a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff400100940 tx=0x7ff3e800a5c0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.895+0000 7ff406ccf640 1 -- 172.21.15.83:0/3304799787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4001010a0 msgr2=0x7ff400198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.895+0000 7ff406ccf640 1 --2- 172.21.15.83:0/3304799787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4001010a0 0x7ff400198670 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff3f802ade0 tx=0x7ff3f8003970 comp rx=0 tx=0).stop 2024-04-25T11:53:43.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.895+0000 7ff406ccf640 1 -- 172.21.15.83:0/3304799787 shutdown_connections 2024-04-25T11:53:43.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.895+0000 7ff406ccf640 1 --2- 172.21.15.83:0/3304799787 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7ff3ec0a3d20 0x7ff3ec0a61e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.895+0000 7ff406ccf640 1 --2- 172.21.15.83:0/3304799787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40019cf30 0x7ff40019d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.895+0000 7ff406ccf640 1 --2- 172.21.15.83:0/3304799787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff400106170 0x7ff400198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.895+0000 7ff406ccf640 1 --2- 172.21.15.83:0/3304799787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4001010a0 0x7ff400198670 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.895+0000 7ff406ccf640 1 -- 172.21.15.83:0/3304799787 >> 172.21.15.83:0/3304799787 conn(0x7ff400077f50 msgr2=0x7ff400069450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:53:43.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.895+0000 7ff406ccf640 1 -- 172.21.15.83:0/3304799787 shutdown_connections 2024-04-25T11:53:43.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:43.895+0000 7ff406ccf640 1 -- 172.21.15.83:0/3304799787 wait complete. 2024-04-25T11:53:43.894 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:37:15.914779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:41:42.546991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:41:43.764226+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:53:43.894 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:53:43.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.007+0000 7f9361944640 1 -- 172.21.15.83:0/736069557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f935c06a860 msgr2=0x7f935c06ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:43.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.007+0000 7f9361944640 1 --2- 172.21.15.83:0/736069557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f935c06a860 0x7f935c06ac40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f934c009900 tx=0x7f934c02a410 comp rx=0 tx=0).stop 2024-04-25T11:53:43.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9361944640 1 -- 172.21.15.83:0/736069557 shutdown_connections 2024-04-25T11:53:43.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9361944640 1 --2- 172.21.15.83:0/736069557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935c107270 0x7f935c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9361944640 1 --2- 172.21.15.83:0/736069557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f935c06b210 0x7f935c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9361944640 1 --2- 172.21.15.83:0/736069557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f935c06a860 0x7f935c06ac40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:43.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9361944640 1 -- 172.21.15.83:0/736069557 >> 172.21.15.83:0/736069557 conn(0x7f935c077eb0 msgr2=0x7f935c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:53:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9361944640 1 -- 172.21.15.83:0/736069557 shutdown_connections 2024-04-25T11:53:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9361944640 1 -- 172.21.15.83:0/736069557 wait complete. 2024-04-25T11:53:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9361944640 1 Processor -- start 2024-04-25T11:53:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9361944640 1 -- start start 2024-04-25T11:53:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9361944640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f935c06a860 0x7f935c19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9361944640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f935c107270 0x7f935c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9361944640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935c1985d0 0x7f935c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9361944640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f935c112230 con 0x7f935c107270 2024-04-25T11:53:43.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9361944640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1120b0 con 0x7f935c1985d0 2024-04-25T11:53:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9361944640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1123b0 con 0x7f935c06a860 2024-04-25T11:53:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.008+0000 7f9353fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f935c107270 0x7f935c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.009+0000 7f9353fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f935c107270 0x7f935c19e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45658/0 (socket says 172.21.15.83:45658) 2024-04-25T11:53:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.009+0000 7f9353fff640 1 -- 172.21.15.83:0/994890158 learned_addr learned my addr 172.21.15.83:0/994890158 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:53:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.009+0000 7f935affd640 1 --2- 172.21.15.83:0/994890158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f935c06a860 0x7f935c19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.009+0000 7f935b7fe640 1 --2- 172.21.15.83:0/994890158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935c1985d0 0x7f935c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.009+0000 7f9353fff640 1 -- 172.21.15.83:0/994890158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f935c06a860 msgr2=0x7f935c19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.009+0000 7f9353fff640 1 --2- 172.21.15.83:0/994890158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f935c06a860 0x7f935c19e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.009+0000 7f9353fff640 1 -- 172.21.15.83:0/994890158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935c1985d0 msgr2=0x7f935c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.009+0000 7f9353fff640 1 --2- 172.21.15.83:0/994890158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935c1985d0 0x7f935c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.009+0000 7f9353fff640 1 -- 172.21.15.83:0/994890158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c1990f0 con 0x7f935c107270 2024-04-25T11:53:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.009+0000 7f9353fff640 1 --2- 172.21.15.83:0/994890158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f935c107270 0x7f935c19e960 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f934400b450 tx=0x7f934400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:53:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.009+0000 7f9360942640 1 -- 172.21.15.83:0/994890158 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9344015020 con 0x7f935c107270 2024-04-25T11:53:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.009+0000 7f9360942640 1 -- 172.21.15.83:0/994890158 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9344011d40 con 0x7f935c107270 2024-04-25T11:53:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.009+0000 7f9361944640 1 -- 172.21.15.83:0/994890158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935c1993e0 con 0x7f935c107270 2024-04-25T11:53:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.009+0000 7f9360942640 1 -- 172.21.15.83:0/994890158 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934400f340 con 0x7f935c107270 2024-04-25T11:53:44.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.009+0000 7f9361944640 1 -- 172.21.15.83:0/994890158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935c1a5630 con 0x7f935c107270 2024-04-25T11:53:44.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.010+0000 7f9361944640 1 -- 172.21.15.83:0/994890158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f935c0691c0 con 0x7f935c107270 2024-04-25T11:53:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.011+0000 7f9360942640 1 -- 172.21.15.83:0/994890158 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9344018020 con 0x7f935c107270 2024-04-25T11:53:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.012+0000 7f9360942640 1 --2- 172.21.15.83:0/994890158 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f93480a3d20 0x7f93480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.012+0000 7f935affd640 1 --2- 172.21.15.83:0/994890158 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f93480a3d20 0x7f93480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.012+0000 7f9360942640 1 -- 172.21.15.83:0/994890158 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f93440d0240 con 0x7f935c107270 2024-04-25T11:53:44.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.012+0000 7f935affd640 1 --2- 172.21.15.83:0/994890158 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f93480a3d20 0x7f93480a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f934c031440 tx=0x7f934c0095f0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:53:44.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.016+0000 7f9360942640 1 -- 172.21.15.83:0/994890158 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f934401f020 con 0x7f935c107270 2024-04-25T11:53:44.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:43 smithi083 ceph-mon[29330]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:53:44.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.193+0000 7f9361944640 1 -- 172.21.15.83:0/994890158 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f935c06b210 con 0x7f935c107270 2024-04-25T11:53:44.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.194+0000 7f9360942640 1 -- 172.21.15.83:0/994890158 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f934400d040 con 0x7f935c107270 2024-04-25T11:53:44.182 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T11:53:44.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.196+0000 7f9361944640 1 -- 172.21.15.83:0/994890158 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f93480a3d20 msgr2=0x7f93480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.196+0000 7f9361944640 1 --2- 172.21.15.83:0/994890158 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f93480a3d20 0x7f93480a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f934c031440 tx=0x7f934c0095f0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.196+0000 7f9361944640 1 -- 172.21.15.83:0/994890158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f935c107270 msgr2=0x7f935c19e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.196+0000 7f9361944640 1 --2- 172.21.15.83:0/994890158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f935c107270 0x7f935c19e960 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f934400b450 tx=0x7f934400b910 comp rx=0 tx=0).stop 2024-04-25T11:53:44.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.196+0000 7f9361944640 1 -- 172.21.15.83:0/994890158 shutdown_connections 2024-04-25T11:53:44.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.196+0000 7f9361944640 1 --2- 172.21.15.83:0/994890158 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f93480a3d20 0x7f93480a61e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.196+0000 7f9361944640 1 --2- 172.21.15.83:0/994890158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f935c107270 0x7f935c19e960 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.196+0000 7f9361944640 1 --2- 172.21.15.83:0/994890158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935c1985d0 0x7f935c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.196+0000 7f9361944640 1 --2- 172.21.15.83:0/994890158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f935c06a860 0x7f935c19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.196+0000 7f9361944640 1 -- 172.21.15.83:0/994890158 >> 172.21.15.83:0/994890158 conn(0x7f935c077eb0 msgr2=0x7f935c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:53:44.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.196+0000 7f9361944640 1 -- 172.21.15.83:0/994890158 shutdown_connections 2024-04-25T11:53:44.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.196+0000 7f9361944640 1 -- 172.21.15.83:0/994890158 wait complete. 2024-04-25T11:53:44.195 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:53:44.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:43 smithi094 ceph-mon[33032]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:53:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.308+0000 7f23bb908640 1 -- 172.21.15.83:0/3289652972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b41006d0 msgr2=0x7f23b4100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.308+0000 7f23bb908640 1 --2- 172.21.15.83:0/3289652972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b41006d0 0x7f23b4100ab0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009900 tx=0x7f23b002a410 comp rx=0 tx=0).stop 2024-04-25T11:53:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.308+0000 7f23bb908640 1 -- 172.21.15.83:0/3289652972 shutdown_connections 2024-04-25T11:53:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.308+0000 7f23bb908640 1 --2- 172.21.15.83:0/3289652972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b41072e0 0x7f23b410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.308+0000 7f23bb908640 1 --2- 172.21.15.83:0/3289652972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b4101080 0x7f23b4106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.308+0000 7f23bb908640 1 --2- 172.21.15.83:0/3289652972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b41006d0 0x7f23b4100ab0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.308+0000 7f23bb908640 1 -- 172.21.15.83:0/3289652972 >> 172.21.15.83:0/3289652972 conn(0x7f23b40773b0 msgr2=0x7f23b40fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:53:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.308+0000 7f23bb908640 1 -- 172.21.15.83:0/3289652972 shutdown_connections 2024-04-25T11:53:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.308+0000 7f23bb908640 1 -- 172.21.15.83:0/3289652972 wait complete. 2024-04-25T11:53:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.308+0000 7f23bb908640 1 Processor -- start 2024-04-25T11:53:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.308+0000 7f23bb908640 1 -- start start 2024-04-25T11:53:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23bb908640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b41006d0 0x7f23b419e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23bb908640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b41072e0 0x7f23b419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23bb908640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b41986b0 0x7f23b4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23bb908640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23b41121d0 con 0x7f23b41072e0 2024-04-25T11:53:44.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23bb908640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f23b4112050 con 0x7f23b41006d0 2024-04-25T11:53:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23bb908640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f23b4112350 con 0x7f23b41986b0 2024-04-25T11:53:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23b8ea3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b41072e0 0x7f23b419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23b8ea3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b41072e0 0x7f23b419ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45672/0 (socket says 172.21.15.83:45672) 2024-04-25T11:53:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23b8ea3640 1 -- 172.21.15.83:0/2030796926 learned_addr learned my addr 172.21.15.83:0/2030796926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:53:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23b9ea5640 1 --2- 172.21.15.83:0/2030796926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b41986b0 0x7f23b4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23b8ea3640 1 -- 172.21.15.83:0/2030796926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b41986b0 msgr2=0x7f23b4198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23b8ea3640 1 --2- 172.21.15.83:0/2030796926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b41986b0 0x7f23b4198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23b8ea3640 1 -- 172.21.15.83:0/2030796926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b41006d0 msgr2=0x7f23b419e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23b8ea3640 1 --2- 172.21.15.83:0/2030796926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b41006d0 0x7f23b419e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23b8ea3640 1 -- 172.21.15.83:0/2030796926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b41992b0 con 0x7f23b41072e0 2024-04-25T11:53:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23b8ea3640 1 --2- 172.21.15.83:0/2030796926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b41072e0 0x7f23b419ea40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f23a400d740 tx=0x7f23a400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:53:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23aa7fc640 1 -- 172.21.15.83:0/2030796926 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a4017070 con 0x7f23b41072e0 2024-04-25T11:53:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23bb908640 1 -- 172.21.15.83:0/2030796926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b41a52a0 con 0x7f23b41072e0 2024-04-25T11:53:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23aa7fc640 1 -- 172.21.15.83:0/2030796926 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23a400de50 con 0x7f23b41072e0 2024-04-25T11:53:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23aa7fc640 1 -- 172.21.15.83:0/2030796926 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a400f620 con 0x7f23b41072e0 2024-04-25T11:53:44.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.309+0000 7f23bb908640 1 -- 172.21.15.83:0/2030796926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b41a5700 con 0x7f23b41072e0 2024-04-25T11:53:44.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.310+0000 7f23bb908640 1 -- 172.21.15.83:0/2030796926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23b4103780 con 0x7f23b41072e0 2024-04-25T11:53:44.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.312+0000 7f23aa7fc640 1 -- 172.21.15.83:0/2030796926 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f23a401a070 con 0x7f23b41072e0 2024-04-25T11:53:44.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.312+0000 7f23aa7fc640 1 --2- 172.21.15.83:0/2030796926 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f23a00a3d20 0x7f23a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:44.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.313+0000 7f23b96a4640 1 --2- 172.21.15.83:0/2030796926 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f23a00a3d20 0x7f23a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:44.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.313+0000 7f23aa7fc640 1 -- 172.21.15.83:0/2030796926 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f23a40d02b0 con 0x7f23b41072e0 2024-04-25T11:53:44.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.313+0000 7f23b96a4640 1 --2- 172.21.15.83:0/2030796926 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f23a00a3d20 0x7f23a00a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f23b00096c0 tx=0x7f23b0031400 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:53:44.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.316+0000 7f23aa7fc640 1 -- 172.21.15.83:0/2030796926 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23a409c950 con 0x7f23b41072e0 2024-04-25T11:53:44.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:43 smithi111 ceph-mon[32784]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-25T11:53:44.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.496+0000 7f23bb908640 1 -- 172.21.15.83:0/2030796926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f23b4101080 con 0x7f23b41072e0 2024-04-25T11:53:44.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.497+0000 7f23aa7fc640 1 -- 172.21.15.83:0/2030796926 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f23a409c250 con 0x7f23b41072e0 2024-04-25T11:53:44.485 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T11:53:44.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.499+0000 7f23bb908640 1 -- 172.21.15.83:0/2030796926 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f23a00a3d20 msgr2=0x7f23a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.499+0000 7f23bb908640 1 --2- 172.21.15.83:0/2030796926 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f23a00a3d20 0x7f23a00a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f23b00096c0 tx=0x7f23b0031400 comp rx=0 tx=0).stop 2024-04-25T11:53:44.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.499+0000 7f23bb908640 1 -- 172.21.15.83:0/2030796926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b41072e0 msgr2=0x7f23b419ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.499+0000 7f23bb908640 1 --2- 172.21.15.83:0/2030796926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b41072e0 0x7f23b419ea40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f23a400d740 tx=0x7f23a400dc00 comp rx=0 tx=0).stop 2024-04-25T11:53:44.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.499+0000 7f23bb908640 1 -- 172.21.15.83:0/2030796926 shutdown_connections 2024-04-25T11:53:44.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.499+0000 7f23bb908640 1 --2- 172.21.15.83:0/2030796926 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f23a00a3d20 0x7f23a00a61e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.499+0000 7f23bb908640 1 --2- 172.21.15.83:0/2030796926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23b41072e0 0x7f23b419ea40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.499+0000 7f23bb908640 1 --2- 172.21.15.83:0/2030796926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f23b41986b0 0x7f23b4198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.499+0000 7f23bb908640 1 --2- 172.21.15.83:0/2030796926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23b41006d0 0x7f23b419e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.499+0000 7f23bb908640 1 -- 172.21.15.83:0/2030796926 >> 172.21.15.83:0/2030796926 conn(0x7f23b40773b0 msgr2=0x7f23b4105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:53:44.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.499+0000 7f23bb908640 1 -- 172.21.15.83:0/2030796926 shutdown_connections 2024-04-25T11:53:44.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.499+0000 7f23bb908640 1 -- 172.21.15.83:0/2030796926 wait complete. 2024-04-25T11:53:44.497 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:53:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.609+0000 7f5d424ef640 1 -- 172.21.15.83:0/2089134044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c103b00 msgr2=0x7f5d3c10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.609+0000 7f5d424ef640 1 --2- 172.21.15.83:0/2089134044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c103b00 0x7f5d3c10bfe0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009900 tx=0x7f5d3002a510 comp rx=0 tx=0).stop 2024-04-25T11:53:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.609+0000 7f5d424ef640 1 -- 172.21.15.83:0/2089134044 shutdown_connections 2024-04-25T11:53:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.609+0000 7f5d424ef640 1 --2- 172.21.15.83:0/2089134044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c103b00 0x7f5d3c10bfe0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.609+0000 7f5d424ef640 1 --2- 172.21.15.83:0/2089134044 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d3c10c520 0x7f5d3c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.609+0000 7f5d424ef640 1 --2- 172.21.15.83:0/2089134044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c1031e0 0x7f5d3c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.609+0000 7f5d424ef640 1 -- 172.21.15.83:0/2089134044 >> 172.21.15.83:0/2089134044 conn(0x7f5d3c0771d0 msgr2=0x7f5d3c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:53:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.609+0000 7f5d424ef640 1 -- 172.21.15.83:0/2089134044 shutdown_connections 2024-04-25T11:53:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.610+0000 7f5d424ef640 1 -- 172.21.15.83:0/2089134044 wait complete. 2024-04-25T11:53:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.610+0000 7f5d424ef640 1 Processor -- start 2024-04-25T11:53:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.610+0000 7f5d424ef640 1 -- start start 2024-04-25T11:53:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.610+0000 7f5d424ef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d3c1031e0 0x7f5d3c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.610+0000 7f5d424ef640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c103b00 0x7f5d3c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:44.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.610+0000 7f5d424ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c10c520 0x7f5d3c1018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:44.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.610+0000 7f5d424ef640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1121b0 con 0x7f5d3c10c520 2024-04-25T11:53:44.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.610+0000 7f5d424ef640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c112030 con 0x7f5d3c103b00 2024-04-25T11:53:44.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.610+0000 7f5d424ef640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c112330 con 0x7f5d3c1031e0 2024-04-25T11:53:44.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d40a8c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c10c520 0x7f5d3c1018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:44.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d40a8c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c10c520 0x7f5d3c1018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45694/0 (socket says 172.21.15.83:45694) 2024-04-25T11:53:44.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d40a8c640 1 -- 172.21.15.83:0/340197396 learned_addr learned my addr 172.21.15.83:0/340197396 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:53:44.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d3b7fe640 1 --2- 172.21.15.83:0/340197396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c103b00 0x7f5d3c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:44.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d3bfff640 1 --2- 172.21.15.83:0/340197396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d3c1031e0 0x7f5d3c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d40a8c640 1 -- 172.21.15.83:0/340197396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d3c1031e0 msgr2=0x7f5d3c100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d40a8c640 1 --2- 172.21.15.83:0/340197396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d3c1031e0 0x7f5d3c100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d40a8c640 1 -- 172.21.15.83:0/340197396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c103b00 msgr2=0x7f5d3c101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d40a8c640 1 --2- 172.21.15.83:0/340197396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c103b00 0x7f5d3c101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d40a8c640 1 -- 172.21.15.83:0/340197396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c1a6c10 con 0x7f5d3c10c520 2024-04-25T11:53:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d40a8c640 1 --2- 172.21.15.83:0/340197396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c10c520 0x7f5d3c1018b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400cad0 tx=0x7f5d3400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:53:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d397fa640 1 -- 172.21.15.83:0/340197396 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400ae20 con 0x7f5d3c10c520 2024-04-25T11:53:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d397fa640 1 -- 172.21.15.83:0/340197396 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d34005020 con 0x7f5d3c10c520 2024-04-25T11:53:44.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d424ef640 1 -- 172.21.15.83:0/340197396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c1a6f00 con 0x7f5d3c10c520 2024-04-25T11:53:44.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d397fa640 1 -- 172.21.15.83:0/340197396 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400f6b0 con 0x7f5d3c10c520 2024-04-25T11:53:44.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.611+0000 7f5d424ef640 1 -- 172.21.15.83:0/340197396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c1a73d0 con 0x7f5d3c10c520 2024-04-25T11:53:44.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.612+0000 7f5d424ef640 1 -- 172.21.15.83:0/340197396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3c0691c0 con 0x7f5d3c10c520 2024-04-25T11:53:44.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.613+0000 7f5d397fa640 1 -- 172.21.15.83:0/340197396 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5d340051d0 con 0x7f5d3c10c520 2024-04-25T11:53:44.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.614+0000 7f5d397fa640 1 --2- 172.21.15.83:0/340197396 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f5d240a3d20 0x7f5d240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:44.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.614+0000 7f5d3bfff640 1 --2- 172.21.15.83:0/340197396 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f5d240a3d20 0x7f5d240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:44.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.614+0000 7f5d397fa640 1 -- 172.21.15.83:0/340197396 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f5d340d07e0 con 0x7f5d3c10c520 2024-04-25T11:53:44.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.615+0000 7f5d3bfff640 1 --2- 172.21.15.83:0/340197396 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f5d240a3d20 0x7f5d240a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c005e10 tx=0x7f5d2c005da0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:53:44.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.619+0000 7f5d397fa640 1 -- 172.21.15.83:0/340197396 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d34014030 con 0x7f5d3c10c520 2024-04-25T11:53:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.798+0000 7f5d424ef640 1 -- 172.21.15.83:0/340197396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5d3c107110 con 0x7f5d3c10c520 2024-04-25T11:53:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.799+0000 7f5d397fa640 1 -- 172.21.15.83:0/340197396 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f5d3409ce80 con 0x7f5d3c10c520 2024-04-25T11:53:44.787 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T11:53:44.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.801+0000 7f5d424ef640 1 -- 172.21.15.83:0/340197396 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f5d240a3d20 msgr2=0x7f5d240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.801+0000 7f5d424ef640 1 --2- 172.21.15.83:0/340197396 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f5d240a3d20 0x7f5d240a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c005e10 tx=0x7f5d2c005da0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.801+0000 7f5d424ef640 1 -- 172.21.15.83:0/340197396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c10c520 msgr2=0x7f5d3c1018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.801+0000 7f5d424ef640 1 --2- 172.21.15.83:0/340197396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c10c520 0x7f5d3c1018b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400cad0 tx=0x7f5d3400cf90 comp rx=0 tx=0).stop 2024-04-25T11:53:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.801+0000 7f5d424ef640 1 -- 172.21.15.83:0/340197396 shutdown_connections 2024-04-25T11:53:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.801+0000 7f5d424ef640 1 --2- 172.21.15.83:0/340197396 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f5d240a3d20 0x7f5d240a61e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.801+0000 7f5d424ef640 1 --2- 172.21.15.83:0/340197396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d3c10c520 0x7f5d3c1018b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.801+0000 7f5d424ef640 1 --2- 172.21.15.83:0/340197396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d3c103b00 0x7f5d3c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.801+0000 7f5d424ef640 1 --2- 172.21.15.83:0/340197396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d3c1031e0 0x7f5d3c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.801+0000 7f5d424ef640 1 -- 172.21.15.83:0/340197396 >> 172.21.15.83:0/340197396 conn(0x7f5d3c0771d0 msgr2=0x7f5d3c0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:53:44.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.801+0000 7f5d424ef640 1 -- 172.21.15.83:0/340197396 shutdown_connections 2024-04-25T11:53:44.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.801+0000 7f5d424ef640 1 -- 172.21.15.83:0/340197396 wait complete. 2024-04-25T11:53:44.799 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T11:53:44.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.910+0000 7f5286d76640 1 -- 172.21.15.83:0/2037891673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280102370 msgr2=0x7f528010dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.910+0000 7f5286d76640 1 --2- 172.21.15.83:0/2037891673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280102370 0x7f528010dd00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5270009930 tx=0x7f527002a4d0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.910+0000 7f5286d76640 1 -- 172.21.15.83:0/2037891673 shutdown_connections 2024-04-25T11:53:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.910+0000 7f5286d76640 1 --2- 172.21.15.83:0/2037891673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280102370 0x7f528010dd00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.910+0000 7f5286d76640 1 --2- 172.21.15.83:0/2037891673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52801019b0 0x7f5280101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.910+0000 7f5286d76640 1 --2- 172.21.15.83:0/2037891673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5280105e90 0x7f5280106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.910+0000 7f5286d76640 1 -- 172.21.15.83:0/2037891673 >> 172.21.15.83:0/2037891673 conn(0x7f5280076130 msgr2=0x7f5280076540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:53:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.910+0000 7f5286d76640 1 -- 172.21.15.83:0/2037891673 shutdown_connections 2024-04-25T11:53:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.910+0000 7f5286d76640 1 -- 172.21.15.83:0/2037891673 wait complete. 2024-04-25T11:53:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.910+0000 7f5286d76640 1 Processor -- start 2024-04-25T11:53:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.910+0000 7f5286d76640 1 -- start start 2024-04-25T11:53:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5286d76640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52801019b0 0x7f5280198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5286d76640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5280102370 0x7f5280198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5286d76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280105e90 0x7f528019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5286d76640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52801110a0 con 0x7f5280105e90 2024-04-25T11:53:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5286d76640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5280110f20 con 0x7f52801019b0 2024-04-25T11:53:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5286d76640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5280111220 con 0x7f5280102370 2024-04-25T11:53:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5285313640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280105e90 0x7f528019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5285313640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280105e90 0x7f528019cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45708/0 (socket says 172.21.15.83:45708) 2024-04-25T11:53:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5285313640 1 -- 172.21.15.83:0/1807452094 learned_addr learned my addr 172.21.15.83:0/1807452094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:53:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5284b12640 1 --2- 172.21.15.83:0/1807452094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52801019b0 0x7f5280198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f527ffff640 1 --2- 172.21.15.83:0/1807452094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5280102370 0x7f5280198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5285313640 1 -- 172.21.15.83:0/1807452094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5280102370 msgr2=0x7f5280198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5285313640 1 --2- 172.21.15.83:0/1807452094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5280102370 0x7f5280198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5285313640 1 -- 172.21.15.83:0/1807452094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52801019b0 msgr2=0x7f5280198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:44.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5285313640 1 --2- 172.21.15.83:0/1807452094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52801019b0 0x7f5280198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:44.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5285313640 1 -- 172.21.15.83:0/1807452094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528019d670 con 0x7f5280105e90 2024-04-25T11:53:44.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5285313640 1 --2- 172.21.15.83:0/1807452094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280105e90 0x7f528019cf40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5270005940 tx=0x7f52700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:53:44.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f527dffb640 1 -- 172.21.15.83:0/1807452094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527003f030 con 0x7f5280105e90 2024-04-25T11:53:44.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f527dffb640 1 -- 172.21.15.83:0/1807452094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5270003cb0 con 0x7f5280105e90 2024-04-25T11:53:44.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f5286d76640 1 -- 172.21.15.83:0/1807452094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528019d900 con 0x7f5280105e90 2024-04-25T11:53:44.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.911+0000 7f527dffb640 1 -- 172.21.15.83:0/1807452094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270035070 con 0x7f5280105e90 2024-04-25T11:53:44.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.912+0000 7f5286d76640 1 -- 172.21.15.83:0/1807452094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52801a51f0 con 0x7f5280105e90 2024-04-25T11:53:44.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.912+0000 7f5286d76640 1 -- 172.21.15.83:0/1807452094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52800691c0 con 0x7f5280105e90 2024-04-25T11:53:44.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.914+0000 7f527dffb640 1 -- 172.21.15.83:0/1807452094 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5270042030 con 0x7f5280105e90 2024-04-25T11:53:44.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.914+0000 7f527dffb640 1 --2- 172.21.15.83:0/1807452094 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f526c0a3d20 0x7f526c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:53:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.914+0000 7f5284b12640 1 --2- 172.21.15.83:0/1807452094 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f526c0a3d20 0x7f526c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:53:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.914+0000 7f527dffb640 1 -- 172.21.15.83:0/1807452094 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8513+0+0 (secure 0 0 0) 0x7f52700f1090 con 0x7f5280105e90 2024-04-25T11:53:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.915+0000 7f5284b12640 1 --2- 172.21.15.83:0/1807452094 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f526c0a3d20 0x7f526c0a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5280077640 tx=0x7f527800a590 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:53:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:44.918+0000 7f527dffb640 1 -- 172.21.15.83:0/1807452094 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52700bd730 con 0x7f5280105e90 2024-04-25T11:53:45.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:45.144+0000 7f5286d76640 1 -- 172.21.15.83:0/1807452094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f528010a3b0 con 0x7f5280105e90 2024-04-25T11:53:45.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:45.153+0000 7f527dffb640 1 -- 172.21.15.83:0/1807452094 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746061 (secure 0 0 0) 0x7f52700bd030 con 0x7f5280105e90 2024-04-25T11:53:45.141 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:53:45.141 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:53:45.141 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 26, 2024-04-25T11:53:45.141 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.142 INFO:teuthology.orchestra.run.smithi083.stdout: "active_gid": 15360, 2024-04-25T11:53:45.142 INFO:teuthology.orchestra.run.smithi083.stdout: "active_name": "x", 2024-04-25T11:53:45.142 INFO:teuthology.orchestra.run.smithi083.stdout: "active_addrs": { 2024-04-25T11:53:45.142 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-25T11:53:45.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.142 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-25T11:53:45.142 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6800", 2024-04-25T11:53:45.142 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4112952368 2024-04-25T11:53:45.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.143 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.143 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-25T11:53:45.143 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-04-25T11:53:45.143 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 4112952368 2024-04-25T11:53:45.143 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.143 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T11:53:45.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.143 INFO:teuthology.orchestra.run.smithi083.stdout: "active_addr": "172.21.15.83:6801/4112952368", 2024-04-25T11:53:45.144 INFO:teuthology.orchestra.run.smithi083.stdout: "active_change": "2024-04-25T11:43:40.200569+0000", 2024-04-25T11:53:45.144 INFO:teuthology.orchestra.run.smithi083.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T11:53:45.144 INFO:teuthology.orchestra.run.smithi083.stdout: "available": true, 2024-04-25T11:53:45.144 INFO:teuthology.orchestra.run.smithi083.stdout: "standbys": [ 2024-04-25T11:53:45.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.144 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 24859, 2024-04-25T11:53:45.144 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "z", 2024-04-25T11:53:45.144 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_features": 4540701547738038271, 2024-04-25T11:53:45.144 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-04-25T11:53:45.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.145 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-04-25T11:53:45.145 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.145 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.145 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.145 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-25T11:53:45.145 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-25T11:53:45.145 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.146 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.146 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.146 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-25T11:53:45.146 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.146 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.146 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.146 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T11:53:45.146 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.146 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.147 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.147 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.147 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.147 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.147 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.147 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.147 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.148 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.148 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.148 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.148 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.148 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.148 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.148 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.148 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.148 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.149 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.149 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.149 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.149 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.149 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.149 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.149 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.149 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.150 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.150 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.150 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.150 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.150 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.150 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.150 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.150 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.151 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.151 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.151 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.151 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.151 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.151 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.151 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.151 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.152 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.152 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.152 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.152 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.152 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.152 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.152 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.153 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.153 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.153 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.153 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.153 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.153 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.153 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.154 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.154 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.154 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.154 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.154 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.154 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.154 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.154 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.154 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.155 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.155 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.155 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.155 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.155 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-04-25T11:53:45.155 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-04-25T11:53:45.155 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.156 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.156 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.156 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.156 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.156 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.156 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.156 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2024-04-25T11:53:45.156 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.157 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.157 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.157 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-04-25T11:53:45.157 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-04-25T11:53:45.157 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.157 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.157 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.157 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-04-25T11:53:45.158 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.158 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.158 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.158 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-04-25T11:53:45.158 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.158 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.158 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.159 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-04-25T11:53:45.159 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-04-25T11:53:45.159 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.159 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.159 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.159 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.159 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.159 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.160 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.160 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-04-25T11:53:45.160 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.160 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.160 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.160 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-04-25T11:53:45.160 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-04-25T11:53:45.160 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.161 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.161 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.161 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.161 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.161 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.161 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.161 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-04-25T11:53:45.161 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.162 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.162 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.162 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-04-25T11:53:45.162 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-04-25T11:53:45.162 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.162 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.162 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.162 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-04-25T11:53:45.163 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.163 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.163 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.163 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-04-25T11:53:45.163 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.163 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.163 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.164 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-04-25T11:53:45.164 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-04-25T11:53:45.164 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.164 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.164 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.164 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.164 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.164 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.165 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.165 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-04-25T11:53:45.165 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.165 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.165 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.165 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-04-25T11:53:45.165 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-04-25T11:53:45.166 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.166 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.166 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.166 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.166 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.166 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.166 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.166 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T11:53:45.166 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.167 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.167 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.167 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-04-25T11:53:45.167 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-04-25T11:53:45.167 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.167 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.167 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.168 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.168 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.168 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.168 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.168 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-04-25T11:53:45.168 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.168 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.168 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.168 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.169 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.169 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.169 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-04-25T11:53:45.169 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.169 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.169 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.169 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-04-25T11:53:45.170 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-04-25T11:53:45.170 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.170 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.170 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.170 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.170 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.170 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.170 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.170 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T11:53:45.171 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.171 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.171 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.171 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-04-25T11:53:45.171 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-04-25T11:53:45.171 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.171 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.172 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.172 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-04-25T11:53:45.172 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.172 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.172 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.172 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T11:53:45.172 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:53:45.172 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.172 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.173 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-04-25T11:53:45.173 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-04-25T11:53:45.173 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-25T11:53:45.173 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.173 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.173 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-25T11:53:45.173 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-25T11:53:45.173 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-25T11:53:45.174 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.174 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T11:53:45.174 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:53:45.174 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.174 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.174 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-04-25T11:53:45.174 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-04-25T11:53:45.175 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-25T11:53:45.175 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.175 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.175 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-04-25T11:53:45.175 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-25T11:53:45.175 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-04-25T11:53:45.175 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.175 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T11:53:45.176 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.176 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.176 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.176 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-04-25T11:53:45.176 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-04-25T11:53:45.176 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.176 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.176 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.177 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T11:53:45.177 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.177 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.177 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.177 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T11:53:45.177 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T11:53:45.177 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.177 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.178 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-04-25T11:53:45.178 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-04-25T11:53:45.178 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:45.178 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.178 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.178 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-04-25T11:53:45.178 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-04-25T11:53:45.178 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-04-25T11:53:45.179 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.179 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-04-25T11:53:45.179 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T11:53:45.179 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.179 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.179 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-04-25T11:53:45.179 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-04-25T11:53:45.180 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.180 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.180 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.180 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-04-25T11:53:45.180 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.180 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.180 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.180 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T11:53:45.180 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:53:45.181 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.181 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.181 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-04-25T11:53:45.181 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-04-25T11:53:45.181 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-25T11:53:45.181 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.181 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.182 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-25T11:53:45.182 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-25T11:53:45.182 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-25T11:53:45.182 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.182 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T11:53:45.182 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:53:45.182 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.182 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.183 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.183 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.183 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.183 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.183 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.183 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.183 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.183 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.183 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.184 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.184 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.184 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.184 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.184 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.184 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.185 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.185 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.185 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.185 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.185 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.185 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.185 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.186 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.186 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.186 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.186 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.188 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.188 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.188 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.188 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.189 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.189 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.189 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.189 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.189 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.189 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.189 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.190 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.190 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.190 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.190 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.190 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.190 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.190 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.190 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.191 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.191 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.191 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.191 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.191 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.191 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.191 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.191 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.192 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.192 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.192 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.192 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.192 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.192 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.192 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.192 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.193 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.193 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.193 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.193 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.193 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.193 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.193 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-04-25T11:53:45.194 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-04-25T11:53:45.194 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:45.194 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.194 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.194 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-25T11:53:45.194 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.194 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.194 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.195 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T11:53:45.195 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.195 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.195 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.195 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-25T11:53:45.195 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-25T11:53:45.195 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.195 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.196 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.196 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-04-25T11:53:45.196 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.196 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.196 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.196 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-04-25T11:53:45.196 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-04-25T11:53:45.196 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-04-25T11:53:45.197 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-04-25T11:53:45.197 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-04-25T11:53:45.197 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.197 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-04-25T11:53:45.197 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.197 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.197 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.197 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-04-25T11:53:45.198 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-04-25T11:53:45.198 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.198 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.198 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.198 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.198 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.198 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.198 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.199 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T11:53:45.199 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.199 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.199 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.199 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-25T11:53:45.199 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-25T11:53:45.199 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.199 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.200 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.200 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-25T11:53:45.200 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.200 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.200 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.200 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T11:53:45.200 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.200 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.200 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.201 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-04-25T11:53:45.201 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-04-25T11:53:45.201 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.201 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.201 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.201 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-25T11:53:45.201 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-25T11:53:45.202 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.202 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.202 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T11:53:45.202 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T11:53:45.202 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.202 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.202 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-04-25T11:53:45.203 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-04-25T11:53:45.203 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-25T11:53:45.203 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.203 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.203 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:45.203 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.203 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.203 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.203 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T11:53:45.204 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.204 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.204 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.204 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.204 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.204 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.204 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-04-25T11:53:45.205 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.205 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.205 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.205 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-04-25T11:53:45.205 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-04-25T11:53:45.205 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:45.205 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.205 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.206 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-25T11:53:45.206 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.206 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.206 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.206 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T11:53:45.206 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.206 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.206 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.207 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-04-25T11:53:45.207 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-04-25T11:53:45.207 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.207 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.207 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.207 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-04-25T11:53:45.207 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.207 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.208 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.208 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T11:53:45.208 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.208 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.208 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.208 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-04-25T11:53:45.208 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-04-25T11:53:45.209 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.209 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.209 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.209 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-04-25T11:53:45.209 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.209 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.209 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.209 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T11:53:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-04-25T11:53:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-04-25T11:53:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.210 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.211 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.211 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.211 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.211 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.211 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.211 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T11:53:45.211 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T11:53:45.211 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.211 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.212 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.212 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-04-25T11:53:45.212 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-04-25T11:53:45.212 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.212 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.212 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.212 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-25T11:53:45.212 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.213 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.213 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.213 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T11:53:45.213 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.213 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.213 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.213 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-04-25T11:53:45.213 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T11:53:45.214 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:45.214 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.214 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.214 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-04-25T11:53:45.214 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.214 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.214 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.214 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T11:53:45.215 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.215 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.215 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.215 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-04-25T11:53:45.215 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-04-25T11:53:45.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.215 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-04-25T11:53:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-04-25T11:53:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-04-25T11:53:45.216 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T11:53:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-04-25T11:53:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-04-25T11:53:45.217 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T11:53:45.218 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-04-25T11:53:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-04-25T11:53:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.219 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T11:53:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.220 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-04-25T11:53:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-04-25T11:53:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.221 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T11:53:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-04-25T11:53:45.222 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-04-25T11:53:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T11:53:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-25T11:53:45.223 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-04-25T11:53:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-04-25T11:53:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.224 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T11:53:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-04-25T11:53:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-04-25T11:53:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-04-25T11:53:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T11:53:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.226 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-04-25T11:53:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-04-25T11:53:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-04-25T11:53:45.227 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T11:53:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-25T11:53:45.228 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-04-25T11:53:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-04-25T11:53:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.229 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T11:53:45.230 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.230 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.230 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.230 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-04-25T11:53:45.230 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.230 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.230 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-04-25T11:53:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-04-25T11:53:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T11:53:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.231 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-04-25T11:53:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-04-25T11:53:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-04-25T11:53:45.232 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T11:53:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-04-25T11:53:45.233 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-04-25T11:53:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-04-25T11:53:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.234 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T11:53:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-04-25T11:53:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-04-25T11:53:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-04-25T11:53:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T11:53:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.236 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-04-25T11:53:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-04-25T11:53:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-04-25T11:53:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.237 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T11:53:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-04-25T11:53:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.238 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-04-25T11:53:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-04-25T11:53:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T11:53:45.239 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-25T11:53:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.240 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-04-25T11:53:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-04-25T11:53:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T11:53:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.241 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-04-25T11:53:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-04-25T11:53:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-04-25T11:53:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.242 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T11:53:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-25T11:53:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.243 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-04-25T11:53:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-04-25T11:53:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T11:53:45.244 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-04-25T11:53:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba": { 2024-04-25T11:53:45.245 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba", 2024-04-25T11:53:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T11:53:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.246 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB container image", 2024-04-25T11:53:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-04-25T11:53:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-04-25T11:53:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.247 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T11:53:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-04-25T11:53:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.248 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-04-25T11:53:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-04-25T11:53:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.249 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T11:53:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-04-25T11:53:45.250 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-04-25T11:53:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-25T11:53:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.251 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T11:53:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-04-25T11:53:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T11:53:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.252 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-04-25T11:53:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T11:53:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.253 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-04-25T11:53:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-04-25T11:53:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-04-25T11:53:45.254 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T11:53:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-04-25T11:53:45.255 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-04-25T11:53:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-04-25T11:53:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-04-25T11:53:45.256 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-04-25T11:53:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-04-25T11:53:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.257 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T11:53:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-04-25T11:53:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-04-25T11:53:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-25T11:53:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.259 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-04-25T11:53:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "grafana_dashboards_path": { 2024-04-25T11:53:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "grafana_dashboards_path", 2024-04-25T11:53:45.260 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T11:53:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T11:53:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.261 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-04-25T11:53:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-04-25T11:53:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.262 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-25T11:53:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-04-25T11:53:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-04-25T11:53:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-04-25T11:53:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.264 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T11:53:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-04-25T11:53:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-04-25T11:53:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.265 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.266 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T11:53:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.270 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.271 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.272 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-04-25T11:53:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-04-25T11:53:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.273 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T11:53:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.274 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.275 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T11:53:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.276 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.277 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.279 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.279 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.279 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.279 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.279 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.279 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.279 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.279 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.280 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.280 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.280 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.280 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.280 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.280 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T11:53:45.280 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T11:53:45.280 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.281 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.281 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.281 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.281 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.281 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.281 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.281 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T11:53:45.281 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.281 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.282 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.282 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T11:53:45.282 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T11:53:45.282 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.282 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.282 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.282 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-04-25T11:53:45.283 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.283 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.283 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.283 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T11:53:45.283 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.283 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.283 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.284 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-04-25T11:53:45.284 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-04-25T11:53:45.284 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.284 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.284 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.284 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:45.284 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.284 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.284 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.285 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-04-25T11:53:45.285 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.285 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.285 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.285 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-04-25T11:53:45.285 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-04-25T11:53:45.285 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.286 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.286 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.286 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:45.286 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.286 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.286 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.286 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T11:53:45.286 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.286 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.287 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.287 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-04-25T11:53:45.287 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-04-25T11:53:45.287 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.287 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.287 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.287 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.288 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.288 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.288 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.288 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-04-25T11:53:45.288 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.288 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.288 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.289 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-25T11:53:45.289 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-25T11:53:45.289 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.289 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.289 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.289 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-04-25T11:53:45.289 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.289 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.289 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.290 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-04-25T11:53:45.290 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-04-25T11:53:45.290 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.290 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T11:53:45.290 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.290 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.290 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.291 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-04-25T11:53:45.291 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-04-25T11:53:45.291 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.291 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.291 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.291 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-04-25T11:53:45.291 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.291 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.291 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.292 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T11:53:45.292 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.292 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.292 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.292 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-04-25T11:53:45.292 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-04-25T11:53:45.292 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.293 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.293 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.293 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T11:53:45.293 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.293 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.293 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.293 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T11:53:45.293 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.293 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.294 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.294 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.294 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-04-25T11:53:45.294 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-04-25T11:53:45.294 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.294 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.294 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.295 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.295 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.295 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.295 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T11:53:45.295 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.295 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.295 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.295 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-04-25T11:53:45.296 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-04-25T11:53:45.296 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.296 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.296 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.296 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.296 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.296 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.296 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.297 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T11:53:45.297 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.297 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.297 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.297 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-04-25T11:53:45.297 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-04-25T11:53:45.297 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.298 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.298 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.298 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.298 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.298 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.298 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.298 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T11:53:45.298 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.298 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.299 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.299 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-04-25T11:53:45.299 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-04-25T11:53:45.299 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.299 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.299 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.299 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.300 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.300 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.300 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.300 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T11:53:45.300 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.300 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.300 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.300 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-04-25T11:53:45.301 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-04-25T11:53:45.301 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.301 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.301 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.301 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.301 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.301 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.301 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.302 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T11:53:45.302 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.302 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.302 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.302 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-04-25T11:53:45.302 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-04-25T11:53:45.302 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.303 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.303 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.303 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-04-25T11:53:45.303 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.303 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.303 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.303 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-04-25T11:53:45.303 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.303 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.304 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.304 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-04-25T11:53:45.304 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-04-25T11:53:45.304 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.304 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.304 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.304 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.304 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.305 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.305 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.305 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T11:53:45.305 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.305 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.305 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.305 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-04-25T11:53:45.306 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-04-25T11:53:45.306 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.306 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.306 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.306 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.306 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.306 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.306 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.306 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T11:53:45.307 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.307 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.307 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.307 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-04-25T11:53:45.307 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-04-25T11:53:45.307 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.307 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.308 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.308 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.308 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.308 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.308 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.308 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T11:53:45.308 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.308 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.308 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.309 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-04-25T11:53:45.309 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-04-25T11:53:45.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.309 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.309 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.309 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.310 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.310 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.310 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T11:53:45.310 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.310 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.310 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.310 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-04-25T11:53:45.311 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-04-25T11:53:45.311 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.311 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.311 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.311 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.311 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.311 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.311 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.311 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T11:53:45.312 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.312 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.312 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.312 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-04-25T11:53:45.312 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-04-25T11:53:45.312 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.312 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.313 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.313 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.313 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.313 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.313 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.313 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T11:53:45.313 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.313 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.313 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.314 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.314 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.314 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-04-25T11:53:45.314 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.314 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.314 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.315 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.315 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.315 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.315 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.315 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.315 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.315 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.316 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.316 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.316 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.316 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.316 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.316 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.316 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.316 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.316 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.317 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.317 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.317 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.317 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.317 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.317 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.317 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.318 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.318 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.318 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.318 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.318 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.318 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.318 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.318 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.318 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.319 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.319 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.319 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.319 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.319 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.319 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.319 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.320 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.320 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.320 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.320 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.320 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.320 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.320 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.320 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.320 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.321 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.321 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.321 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.321 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.321 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.321 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.322 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.322 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.322 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.322 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.322 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.322 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.322 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.322 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.322 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.323 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.323 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-04-25T11:53:45.323 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-04-25T11:53:45.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.323 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.323 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.323 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-04-25T11:53:45.324 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.324 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.324 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.324 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T11:53:45.324 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.324 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.324 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.324 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-04-25T11:53:45.325 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-04-25T11:53:45.325 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.325 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.325 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.325 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-04-25T11:53:45.325 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.325 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.325 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.325 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T11:53:45.326 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.326 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.326 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.326 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.326 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.326 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.326 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-04-25T11:53:45.327 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.327 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.327 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.327 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T11:53:45.327 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T11:53:45.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.327 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.327 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.327 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:45.328 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.328 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.328 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.328 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.328 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.328 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.328 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.329 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T11:53:45.329 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T11:53:45.329 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.329 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.329 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.329 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.329 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.329 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.329 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.330 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.330 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.330 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.330 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.330 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T11:53:45.330 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T11:53:45.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.331 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.331 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.331 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.331 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.331 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.331 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.331 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.331 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.331 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-04-25T11:53:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T11:53:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.332 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.333 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T11:53:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T11:53:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.334 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T11:53:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T11:53:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.335 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.336 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T11:53:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T11:53:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.337 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.338 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T11:53:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T11:53:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.339 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T11:53:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T11:53:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.340 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.341 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T11:53:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T11:53:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.342 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.343 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T11:53:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T11:53:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.344 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T11:53:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T11:53:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.345 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.346 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T11:53:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T11:53:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.347 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.348 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T11:53:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T11:53:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.349 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.350 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.350 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.350 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.350 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T11:53:45.350 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T11:53:45.350 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.350 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.351 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.351 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-25T11:53:45.351 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.351 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.351 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.351 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.351 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.351 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.351 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T11:53:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T11:53:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.352 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-04-25T11:53:45.353 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-04-25T11:53:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.354 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T11:53:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T11:53:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.355 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-25T11:53:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T11:53:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T11:53:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.357 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T11:53:45.358 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T11:53:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.359 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T11:53:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T11:53:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.360 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.361 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T11:53:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T11:53:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.362 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T11:53:45.363 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T11:53:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "[]", 2024-04-25T11:53:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.364 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T11:53:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T11:53:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.365 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "{}", 2024-04-25T11:53:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T11:53:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T11:53:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.367 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T11:53:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T11:53:45.368 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.369 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T11:53:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T11:53:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.370 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T11:53:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T11:53:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.372 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T11:53:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T11:53:45.373 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.374 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T11:53:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T11:53:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.375 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.376 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T11:53:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T11:53:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.377 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T11:53:45.378 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T11:53:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.379 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T11:53:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T11:53:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.380 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.381 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T11:53:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T11:53:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.382 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T11:53:45.383 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T11:53:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T11:53:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.384 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T11:53:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T11:53:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.385 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T11:53:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T11:53:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-04-25T11:53:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.387 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T11:53:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T11:53:45.388 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-04-25T11:53:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.389 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T11:53:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T11:53:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.390 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T11:53:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T11:53:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-25T11:53:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.392 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T11:53:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T11:53:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.393 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.394 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T11:53:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T11:53:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.395 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.396 INFO:teuthology.orchestra.run.smithi083.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T11:53:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T11:53:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.397 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T11:53:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T11:53:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.398 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-25T11:53:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.399 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T11:53:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T11:53:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:45.400 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T11:53:45.401 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T11:53:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-25T11:53:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.402 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-04-25T11:53:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-04-25T11:53:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.403 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.404 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-04-25T11:53:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-04-25T11:53:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.405 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.406 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-04-25T11:53:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-04-25T11:53:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.407 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-04-25T11:53:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-04-25T11:53:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-04-25T11:53:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.408 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-04-25T11:53:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.409 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-25T11:53:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-25T11:53:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.410 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.411 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.412 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.413 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.414 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.415 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.416 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.417 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.418 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.419 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-04-25T11:53:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-04-25T11:53:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.420 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-04-25T11:53:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.421 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-04-25T11:53:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T11:53:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.422 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-25T11:53:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-25T11:53:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.423 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-25T11:53:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.424 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-25T11:53:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-25T11:53:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-04-25T11:53:45.425 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.426 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-25T11:53:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-25T11:53:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.427 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-04-25T11:53:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-04-25T11:53:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.428 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-04-25T11:53:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.429 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-25T11:53:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-25T11:53:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-04-25T11:53:45.430 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-04-25T11:53:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.431 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-25T11:53:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-25T11:53:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-25T11:53:45.432 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-25T11:53:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-25T11:53:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-04-25T11:53:45.433 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-04-25T11:53:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.434 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.435 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-04-25T11:53:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-04-25T11:53:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-04-25T11:53:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.436 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.437 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.437 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.437 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.437 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-04-25T11:53:45.437 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.437 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.437 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.438 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.438 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.438 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.438 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.438 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.438 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.438 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.438 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.439 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.439 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.439 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.439 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.439 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.439 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.439 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.439 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.439 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.440 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.440 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.440 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.440 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.440 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.440 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.440 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.441 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.441 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.441 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.441 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.441 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.441 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.441 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.441 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.442 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.442 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.442 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.442 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.442 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.442 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.442 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.442 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.443 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.443 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.443 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.443 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.443 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.443 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.443 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.443 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.443 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.444 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.444 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.444 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.444 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.444 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.444 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.444 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.444 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.445 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.445 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.445 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.445 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.445 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.445 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.445 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.445 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.446 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.446 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-04-25T11:53:45.446 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-04-25T11:53:45.446 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.446 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.446 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.446 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-04-25T11:53:45.447 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.447 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.447 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.447 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T11:53:45.447 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.447 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.447 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.447 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-04-25T11:53:45.448 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-04-25T11:53:45.448 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.448 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.448 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.448 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-04-25T11:53:45.448 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.448 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.448 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.449 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T11:53:45.449 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.449 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.449 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.449 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.449 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-04-25T11:53:45.449 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-04-25T11:53:45.449 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.449 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.450 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.450 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-04-25T11:53:45.450 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.450 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.450 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.450 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-04-25T11:53:45.450 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.450 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.451 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.451 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-04-25T11:53:45.451 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-04-25T11:53:45.451 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.451 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.451 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.451 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-25T11:53:45.451 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.452 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.452 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.452 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T11:53:45.452 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.452 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.452 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.452 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-04-25T11:53:45.452 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-04-25T11:53:45.453 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.453 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.453 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.453 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.453 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.453 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.453 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.453 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T11:53:45.454 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.454 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.454 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.454 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-25T11:53:45.454 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-25T11:53:45.454 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.454 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.454 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.455 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-25T11:53:45.455 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.455 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.455 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.455 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T11:53:45.455 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.455 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.455 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.456 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-04-25T11:53:45.456 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-04-25T11:53:45.456 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.456 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.456 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.456 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-04-25T11:53:45.456 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.456 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.457 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.457 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T11:53:45.457 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.457 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.457 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.457 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.457 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.458 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.458 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-04-25T11:53:45.458 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.458 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.458 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.458 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.458 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.458 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.458 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.459 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.459 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.459 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.459 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.459 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.459 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.459 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.459 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.460 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.460 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.460 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.460 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.460 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.460 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.460 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.460 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.461 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.461 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.461 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.461 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.461 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.461 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.461 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.461 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.461 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.462 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.462 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.462 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.462 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.462 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.462 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.462 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.462 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.463 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.463 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.463 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.463 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.463 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.463 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.463 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.463 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.463 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.464 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.464 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.464 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.464 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.464 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.464 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.464 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.464 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.465 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.465 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.465 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.465 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.465 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.465 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.465 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.465 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.466 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.466 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.466 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.466 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.466 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.466 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-04-25T11:53:45.466 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-04-25T11:53:45.467 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.467 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.467 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.467 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-25T11:53:45.467 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.467 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.467 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.467 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.467 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.468 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.468 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.468 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-04-25T11:53:45.468 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-04-25T11:53:45.468 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.468 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.468 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.469 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-04-25T11:53:45.469 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.469 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.469 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.469 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.469 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.469 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.469 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.470 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-25T11:53:45.470 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-25T11:53:45.470 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.470 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.470 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.470 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-25T11:53:45.470 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.470 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.471 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.471 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.471 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.471 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.471 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.471 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.471 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.472 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-04-25T11:53:45.472 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-04-25T11:53:45.474 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-04-25T11:53:45.474 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.474 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-04-25T11:53:45.474 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-04-25T11:53:45.474 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.475 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.475 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.475 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-04-25T11:53:45.475 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.475 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.475 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.475 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T11:53:45.475 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.476 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.476 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.476 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-04-25T11:53:45.476 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-04-25T11:53:45.476 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.476 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.476 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.476 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-04-25T11:53:45.477 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.477 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.477 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.477 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T11:53:45.477 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.477 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.477 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.478 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-04-25T11:53:45.478 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-04-25T11:53:45.478 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.478 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.478 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.478 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.478 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.478 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.479 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.479 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-04-25T11:53:45.479 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.479 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.479 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.479 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-25T11:53:45.479 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-25T11:53:45.479 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.480 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.480 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.480 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-04-25T11:53:45.480 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-04-25T11:53:45.480 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.480 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.480 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T11:53:45.480 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.481 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.481 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.481 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.481 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.481 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.481 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.482 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.482 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.482 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.482 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.482 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.482 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.482 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.482 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.483 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.483 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.483 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.483 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.483 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.483 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.483 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.483 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.484 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.484 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.484 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.484 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.484 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.484 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.484 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.484 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.485 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.485 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.485 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.485 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.485 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.485 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.485 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.485 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.486 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.486 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.486 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.486 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.486 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.486 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.486 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.486 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.487 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.487 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.487 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.487 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.487 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.487 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.487 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.487 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.487 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.488 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.488 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.488 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.488 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.488 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.488 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.488 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.489 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.489 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.489 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.489 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.489 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.489 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.489 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-04-25T11:53:45.489 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-04-25T11:53:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.490 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-04-25T11:53:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-04-25T11:53:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-04-25T11:53:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.491 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-04-25T11:53:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-04-25T11:53:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.492 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.493 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-25T11:53:45.493 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-25T11:53:45.493 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.493 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.494 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:44 smithi111 ceph-mon[32784]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:53:45.494 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:44 smithi111 ceph-mon[32784]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:53:45.494 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:44 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3304799787' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:53:45.495 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:44 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/994890158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:53:45.495 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:44 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2030796926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:53:45.495 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:44 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/340197396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:53:45.495 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:44 smithi094 ceph-mon[33032]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:53:45.495 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:44 smithi094 ceph-mon[33032]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:53:45.495 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:44 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3304799787' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:53:45.495 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:44 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/994890158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:53:45.495 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:44 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2030796926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:53:45.496 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:44 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/340197396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:53:45.499 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.499 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-04-25T11:53:45.499 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:53:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-04-25T11:53:45.500 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-04-25T11:53:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-25T11:53:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-25T11:53:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-04-25T11:53:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.501 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T11:53:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-04-25T11:53:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-04-25T11:53:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.502 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-04-25T11:53:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.503 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-04-25T11:53:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-04-25T11:53:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-04-25T11:53:45.504 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:53:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-04-25T11:53:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.506 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.507 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.508 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.508 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.508 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.508 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.508 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.508 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.508 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.508 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.509 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.509 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.509 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.509 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.509 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.509 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.509 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.509 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.510 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.510 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.510 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.510 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.510 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.510 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.510 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.510 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.511 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.512 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.513 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.514 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.515 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.515 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-04-25T11:53:45.515 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.515 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.515 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.515 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.515 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.515 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.516 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.516 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.516 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.516 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.516 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.516 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.516 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.516 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.516 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.517 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.517 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.517 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.517 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.517 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.517 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.517 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.517 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.518 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.518 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.518 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.518 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.518 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.518 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.518 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.518 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.519 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.519 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.519 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.519 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.519 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.519 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.519 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.520 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.520 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.520 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.520 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.520 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.520 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.520 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.520 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.520 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.521 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.521 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.521 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.521 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.521 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.521 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.521 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.521 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.522 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.522 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.522 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.522 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.522 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.522 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.522 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.523 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.523 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.523 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.523 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.523 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.523 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.523 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.523 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.523 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.524 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.524 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-04-25T11:53:45.524 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.524 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.524 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.524 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-04-25T11:53:45.524 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-04-25T11:53:45.525 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.525 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.525 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.525 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-04-25T11:53:45.525 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.525 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.525 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.525 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T11:53:45.525 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.526 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.526 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.526 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-04-25T11:53:45.526 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-04-25T11:53:45.526 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.526 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.526 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.527 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:45.527 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-04-25T11:53:45.527 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.527 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.527 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T11:53:45.527 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.527 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.527 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.528 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.528 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.528 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.528 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.528 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.528 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.528 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.528 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.529 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.529 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.529 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.529 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.529 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.529 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.529 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.529 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.529 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.530 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.530 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.530 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.530 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.530 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.530 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.530 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.530 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.531 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.531 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.531 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.531 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.531 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.531 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.531 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.531 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.532 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.532 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.532 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.532 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.532 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.532 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.532 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.532 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.533 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.533 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.533 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.533 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.533 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.533 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.533 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.533 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.533 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.534 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.535 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-04-25T11:53:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.536 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-04-25T11:53:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-04-25T11:53:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-04-25T11:53:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.537 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-04-25T11:53:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T11:53:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.538 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.539 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.541 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.542 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.543 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.544 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.545 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-04-25T11:53:45.546 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-04-25T11:53:45.547 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.547 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.547 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.547 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.547 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.547 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.547 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.547 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-04-25T11:53:45.548 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T11:53:45.548 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.548 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.548 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-04-25T11:53:45.548 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-04-25T11:53:45.548 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.548 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.548 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.549 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-04-25T11:53:45.549 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.549 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.549 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.549 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-04-25T11:53:45.549 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.549 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.549 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.550 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-04-25T11:53:45.550 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-04-25T11:53:45.550 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.550 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.550 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.550 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-04-25T11:53:45.550 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.550 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.551 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.551 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T11:53:45.551 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.551 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.551 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.551 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-04-25T11:53:45.551 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-04-25T11:53:45.551 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.552 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.552 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.552 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.552 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.552 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.552 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.552 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-04-25T11:53:45.552 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.553 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.553 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.553 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-04-25T11:53:45.553 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-04-25T11:53:45.553 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.553 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.553 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.553 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-04-25T11:53:45.554 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.554 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.554 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.554 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T11:53:45.554 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T11:53:45.554 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.554 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.554 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.555 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.555 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.555 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-04-25T11:53:45.555 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.555 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.555 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.555 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.555 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.556 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.556 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.556 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.556 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.556 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.556 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.556 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.556 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.557 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.557 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.557 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.557 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.557 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.557 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.557 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.557 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.557 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.558 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.558 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.558 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.558 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.558 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.558 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.558 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.559 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.559 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.559 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.559 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.559 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.559 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.559 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.559 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.560 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.560 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.560 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.560 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.560 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.560 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.560 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.560 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.561 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.561 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.561 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.561 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.561 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.561 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.561 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.561 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.561 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.562 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.562 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.562 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.562 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.562 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.562 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.562 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.563 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.563 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.563 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.563 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.563 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.563 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.563 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.563 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.563 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.564 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.564 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.564 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-04-25T11:53:45.564 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.564 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.564 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.564 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.564 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.565 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.565 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.565 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.565 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.565 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.565 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.565 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.565 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.566 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.566 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.566 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.566 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.566 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.566 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.566 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.566 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.567 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.567 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.567 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.567 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.567 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.567 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.567 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.567 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.568 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.568 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.568 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.568 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.568 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.568 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.568 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.568 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.569 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.569 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.569 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.569 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.569 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.569 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.569 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.569 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.570 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.570 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.570 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.570 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.570 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.570 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.570 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.570 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.570 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.571 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.571 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.571 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.571 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.571 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.571 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.571 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.571 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.572 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.572 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.572 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.572 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.572 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.572 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.572 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.572 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.572 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.573 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.573 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.573 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.573 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-04-25T11:53:45.573 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.573 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.573 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.573 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.573 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.574 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.574 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.574 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.574 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.574 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.574 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.574 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.574 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.575 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.575 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.575 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.575 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.575 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.575 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.575 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.575 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.575 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.576 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.576 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.576 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.576 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.576 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.576 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.576 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.577 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.577 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.577 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.577 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.577 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.577 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.577 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.577 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.578 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.578 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.578 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.578 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.578 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.578 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.578 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.578 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.579 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.579 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.579 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.579 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.579 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.579 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.579 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.579 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.580 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.580 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.580 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.580 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.580 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.580 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.580 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.581 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.581 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.581 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.581 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.581 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.581 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.581 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.581 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.581 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.582 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.582 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.582 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-25T11:53:45.582 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.582 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.582 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.583 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-04-25T11:53:45.583 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-04-25T11:53:45.583 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.583 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.583 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.583 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.583 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.583 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.583 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.584 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T11:53:45.584 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.584 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.584 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.584 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.584 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.584 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.584 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.585 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.585 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.585 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.585 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.585 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.585 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.585 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.585 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.585 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.586 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.586 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.586 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.586 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.586 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.586 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.586 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.586 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.587 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.587 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.587 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.587 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.587 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.587 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.587 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.587 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.588 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.588 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.588 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.588 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.588 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.588 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.588 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.589 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.590 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.591 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.592 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-04-25T11:53:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-25T11:53:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.593 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-04-25T11:53:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-04-25T11:53:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-04-25T11:53:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-04-25T11:53:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.594 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-04-25T11:53:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.595 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.596 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.597 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.598 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.600 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.601 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.602 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.603 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.603 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.603 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.603 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.603 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.603 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.603 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.603 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.603 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-04-25T11:53:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.604 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.605 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.606 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.607 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.608 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.609 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.610 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.611 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.612 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-04-25T11:53:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.613 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.614 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.615 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.616 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.619 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.620 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-25T11:53:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-25T11:53:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-25T11:53:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.622 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-04-25T11:53:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-04-25T11:53:45.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-25T11:53:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-04-25T11:53:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-04-25T11:53:45.624 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T11:53:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-04-25T11:53:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.625 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-04-25T11:53:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-04-25T11:53:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.626 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T11:53:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-25T11:53:45.627 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-25T11:53:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.628 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.629 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.630 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.631 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.632 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.633 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.634 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.636 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-04-25T11:53:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-04-25T11:53:45.637 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-04-25T11:53:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-04-25T11:53:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.638 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-25T11:53:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-25T11:53:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.639 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-25T11:53:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-04-25T11:53:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.640 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-04-25T11:53:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-04-25T11:53:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-04-25T11:53:45.641 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.642 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-04-25T11:53:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-04-25T11:53:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.643 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T11:53:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T11:53:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.645 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.646 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.647 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.648 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.649 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.650 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.651 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.652 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-04-25T11:53:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-04-25T11:53:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.653 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.654 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T11:53:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T11:53:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-04-25T11:53:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.655 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-04-25T11:53:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-04-25T11:53:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:45.656 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-04-25T11:53:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.657 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-25T11:53:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-25T11:53:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-25T11:53:45.658 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T11:53:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.659 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-25T11:53:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-25T11:53:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-04-25T11:53:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.660 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T11:53:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-04-25T11:53:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-04-25T11:53:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.661 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-04-25T11:53:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.662 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-25T11:53:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-25T11:53:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-04-25T11:53:45.663 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-04-25T11:53:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-04-25T11:53:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.664 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-25T11:53:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-25T11:53:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-25T11:53:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-25T11:53:45.665 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-25T11:53:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-04-25T11:53:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.667 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.668 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.670 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.671 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.672 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.673 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.674 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.675 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.678 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.679 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-04-25T11:53:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-04-25T11:53:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:45.680 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.681 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-04-25T11:53:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T11:53:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.682 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-04-25T11:53:45.683 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-04-25T11:53:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.684 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.685 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-04-25T11:53:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-04-25T11:53:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-04-25T11:53:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.686 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.687 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-25T11:53:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-25T11:53:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.688 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.689 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.690 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.691 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.692 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.693 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.695 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.696 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.697 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.698 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-25T11:53:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-25T11:53:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.699 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-25T11:53:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-25T11:53:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.701 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.702 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-04-25T11:53:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.703 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.704 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.705 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.706 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.707 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.708 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.709 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.710 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.711 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.712 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-04-25T11:53:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.713 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.714 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.715 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.716 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.718 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.719 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.720 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.721 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.722 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-04-25T11:53:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-04-25T11:53:45.723 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-04-25T11:53:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.724 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T11:53:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-04-25T11:53:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.726 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.727 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.727 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.727 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.727 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.727 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.728 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.728 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.728 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.728 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.728 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.728 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.729 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.729 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.729 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.729 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.729 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.729 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.729 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.730 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.730 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.730 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.730 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.730 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.730 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.730 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.731 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.731 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.731 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.731 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.731 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.731 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.731 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.731 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.732 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.732 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.732 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.732 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.732 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.732 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.732 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.732 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.733 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.733 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.733 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.733 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.733 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.733 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.733 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.733 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.733 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.734 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.734 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.734 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.734 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.734 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.735 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.735 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.735 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.735 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.735 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.735 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.735 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-04-25T11:53:45.735 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-04-25T11:53:45.736 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.736 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.736 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.736 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.736 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.736 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.736 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.736 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.737 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.737 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.737 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.737 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-04-25T11:53:45.737 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-04-25T11:53:45.737 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.737 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.738 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.738 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-04-25T11:53:45.738 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.738 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.738 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.738 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.738 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.739 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.739 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.739 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-04-25T11:53:45.739 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-04-25T11:53:45.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.739 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.739 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.739 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.740 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.740 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.740 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.740 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.740 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.740 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.740 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.741 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-04-25T11:53:45.741 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-04-25T11:53:45.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.741 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.741 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.741 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.741 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.742 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.742 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.742 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.742 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.742 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.742 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.743 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-04-25T11:53:45.743 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-04-25T11:53:45.743 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:45.743 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.743 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.743 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.743 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.743 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.744 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.744 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.744 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.744 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.744 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.744 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-04-25T11:53:45.744 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-04-25T11:53:45.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.745 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.745 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.745 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.745 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.745 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.745 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.745 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.746 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.746 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.746 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.746 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-04-25T11:53:45.746 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-04-25T11:53:45.746 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.746 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.747 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.747 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.747 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.747 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.747 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.747 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.747 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.747 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.747 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.748 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-04-25T11:53:45.748 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-04-25T11:53:45.748 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.748 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.748 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.748 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.748 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.748 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.749 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.749 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.749 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.749 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.749 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.749 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-04-25T11:53:45.749 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-04-25T11:53:45.749 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.750 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.750 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.750 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.750 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-25T11:53:45.750 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-04-25T11:53:45.750 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.750 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.750 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.751 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.751 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.751 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-04-25T11:53:45.751 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-04-25T11:53:45.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.751 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.751 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.752 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.752 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.752 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.752 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.752 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.752 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.753 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.753 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-04-25T11:53:45.753 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-04-25T11:53:45.753 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.753 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.753 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.754 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.754 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.754 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.754 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.754 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.754 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.755 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.755 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.755 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-04-25T11:53:45.755 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-04-25T11:53:45.755 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.755 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.755 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.756 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.756 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.756 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.756 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.756 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.756 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.756 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.757 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.757 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.757 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.757 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.757 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-04-25T11:53:45.757 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.758 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.758 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.758 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-04-25T11:53:45.758 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-04-25T11:53:45.758 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.758 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.759 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.759 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.759 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.759 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.759 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.759 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T11:53:45.760 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.760 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.760 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.760 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-04-25T11:53:45.760 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-04-25T11:53:45.761 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.761 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.761 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.761 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.761 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.762 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.762 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-04-25T11:53:45.762 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.762 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.762 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.763 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.763 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.763 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.763 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.763 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.763 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.764 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.764 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.764 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.765 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.765 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.765 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.765 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.765 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.766 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.766 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.766 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.766 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.766 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.766 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.767 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.767 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.767 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.767 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.767 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.767 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.767 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.767 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.768 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.769 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.769 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.769 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.769 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.769 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.769 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.769 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.769 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.769 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.770 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.770 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.770 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.770 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.770 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.770 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.770 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.770 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.770 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.771 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.771 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.771 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.771 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.771 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.771 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.771 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.772 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.772 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.772 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.772 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.772 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.772 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.772 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.773 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.773 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.773 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-04-25T11:53:45.773 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.773 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.773 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.773 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.774 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.774 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.774 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.774 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.774 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.774 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.774 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.775 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.775 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.775 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.775 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.775 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.775 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.775 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.775 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.775 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.776 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.776 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.776 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.776 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.776 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.776 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.777 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.777 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.777 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.777 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.777 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.777 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.777 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.777 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.778 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.778 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.778 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.778 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.778 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.778 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.779 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.779 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.779 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.779 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.779 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.779 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.779 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.780 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.780 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.780 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.780 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.780 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.780 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.780 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.780 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.780 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.781 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.781 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.781 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.781 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.781 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.781 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.781 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.781 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.782 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.782 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.782 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.782 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.782 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.782 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.783 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.783 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-04-25T11:53:45.783 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.783 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.783 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.783 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.783 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.783 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.783 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.784 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.784 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.784 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.784 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.784 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.784 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.784 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.784 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.784 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.785 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.785 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.785 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.785 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.785 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.785 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.785 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.786 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.786 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.786 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.786 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.786 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.786 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.786 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.786 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.787 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.787 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.787 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.787 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.787 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.787 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.787 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.788 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.788 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.788 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.788 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.788 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.788 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.789 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.789 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.789 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.789 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.789 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.789 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.789 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.789 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.789 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.790 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.790 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.790 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.790 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.790 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.791 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.791 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.791 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.791 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.791 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.791 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.791 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.791 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.791 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.792 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.792 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.792 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.792 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.792 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.792 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-04-25T11:53:45.793 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.793 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.793 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.793 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-04-25T11:53:45.793 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-04-25T11:53:45.793 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.793 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.793 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.794 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T11:53:45.794 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.794 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.794 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.794 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.794 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.794 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.795 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.795 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-25T11:53:45.795 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-25T11:53:45.795 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.795 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.795 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.795 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-04-25T11:53:45.796 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.796 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.796 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.796 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.796 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.796 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.796 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.797 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.797 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.797 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.797 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.797 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.797 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.797 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.798 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.798 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.798 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.798 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.798 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.798 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.799 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.799 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.799 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.799 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.799 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.799 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.799 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.800 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.800 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.800 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.800 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.800 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.800 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.800 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.801 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.801 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.801 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.801 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.801 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.801 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.801 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.802 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.802 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.802 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.802 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.802 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.802 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.802 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.802 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.802 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.803 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.803 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.803 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.803 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.803 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.803 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.803 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.803 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.804 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.804 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.804 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.804 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.804 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.804 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.804 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.804 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.805 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.805 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.805 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.805 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.805 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.805 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.805 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.805 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.805 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.806 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-04-25T11:53:45.806 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.806 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.806 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.806 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-04-25T11:53:45.806 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-04-25T11:53:45.806 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.807 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.807 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.807 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.807 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.807 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.807 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.807 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T11:53:45.807 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.808 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.808 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.808 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-04-25T11:53:45.808 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-04-25T11:53:45.808 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.808 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.808 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.808 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.809 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.809 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.809 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.809 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T11:53:45.809 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.809 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.810 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.810 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-04-25T11:53:45.810 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-04-25T11:53:45.810 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.810 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.810 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.810 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.811 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.811 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.811 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.811 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T11:53:45.811 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.811 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.811 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.812 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-04-25T11:53:45.812 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-04-25T11:53:45.812 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.812 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.812 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.812 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.812 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.812 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.813 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.813 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T11:53:45.813 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.813 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.813 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.813 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-04-25T11:53:45.814 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-04-25T11:53:45.814 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.814 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.814 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.814 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.814 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.814 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.814 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.815 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T11:53:45.815 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.815 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.815 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.815 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.815 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-04-25T11:53:45.815 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-04-25T11:53:45.815 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.815 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.816 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.816 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.816 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.816 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.816 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.816 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.816 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.816 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.817 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.817 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-04-25T11:53:45.817 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-04-25T11:53:45.817 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.817 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.817 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.817 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.818 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.818 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.818 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.818 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.818 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.818 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.818 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.819 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-04-25T11:53:45.819 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-04-25T11:53:45.819 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.819 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.819 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.819 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T11:53:45.819 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.820 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.820 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.820 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.820 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.820 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.820 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.820 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-25T11:53:45.821 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-25T11:53:45.821 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.821 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.821 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.821 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.821 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.821 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.821 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.821 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.822 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.822 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.822 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.822 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-25T11:53:45.822 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-25T11:53:45.822 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.822 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.823 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.823 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-04-25T11:53:45.823 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-04-25T11:53:45.823 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.823 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.823 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.823 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.823 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.824 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.824 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-04-25T11:53:45.824 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-04-25T11:53:45.824 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.824 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.824 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.824 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-04-25T11:53:45.824 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.825 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.825 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.825 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.825 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.825 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.825 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.825 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-04-25T11:53:45.825 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-04-25T11:53:45.826 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.826 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.826 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.826 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.826 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.826 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.826 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.826 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.827 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.827 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.827 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.827 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.827 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-04-25T11:53:45.827 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-04-25T11:53:45.827 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.827 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.828 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.828 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.828 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.828 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.828 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.828 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.828 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.828 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.828 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.829 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.829 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.829 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.829 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.829 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.829 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.829 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.830 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.830 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.830 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.830 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.830 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.830 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.830 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.830 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.831 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.831 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.831 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.831 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.831 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.831 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.831 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.832 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.832 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.832 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.832 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.832 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.832 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.832 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.832 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.832 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.833 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.833 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.833 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.833 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.833 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.833 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.833 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.833 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.833 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.834 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.834 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.834 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.834 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.834 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.834 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.834 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.834 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.835 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.835 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.835 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.835 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.835 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.835 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.835 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.836 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.836 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.836 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.836 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.836 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.836 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.836 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.836 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.837 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.837 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.837 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.837 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.837 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-04-25T11:53:45.837 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-04-25T11:53:45.837 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.837 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.838 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.838 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.838 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.838 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.838 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.838 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.838 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.838 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.839 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.839 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-04-25T11:53:45.839 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-04-25T11:53:45.839 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.839 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.839 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.839 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.839 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.840 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.840 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.840 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.840 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.840 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.840 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.840 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-04-25T11:53:45.841 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-04-25T11:53:45.841 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.841 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.841 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.841 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T11:53:45.841 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.841 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.841 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.842 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.842 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.842 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.842 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.842 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.842 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.842 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.842 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-04-25T11:53:45.843 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.843 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.843 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.843 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.843 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.843 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.843 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.843 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.844 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.844 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.844 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.844 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.844 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.844 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.844 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.844 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.845 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.845 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.845 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.845 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.845 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.845 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.845 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.845 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.846 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.846 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.846 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.846 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.846 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.846 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.846 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.846 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.847 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.847 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.847 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.847 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.847 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.847 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.847 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.848 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.848 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.848 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.848 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.848 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.848 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.848 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.848 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.848 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.849 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.849 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.849 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.849 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.849 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.849 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.849 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.849 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.850 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.850 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.850 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.850 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.850 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.850 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.850 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.850 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.851 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.851 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.851 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.851 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.851 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.851 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.851 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.852 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-04-25T11:53:45.852 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.852 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.852 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.852 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.852 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.852 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.852 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.853 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.853 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.853 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.853 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.853 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.853 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.853 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.853 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.854 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.854 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.854 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.854 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.854 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.854 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.854 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.854 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.855 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.855 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.855 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.855 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.855 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.855 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.855 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.855 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.856 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.856 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.856 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.856 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.856 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.856 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.856 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.856 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.857 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.857 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.857 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.857 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.857 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.857 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.857 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.857 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.858 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.858 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.858 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.858 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.858 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.858 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.858 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.858 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.859 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.859 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.859 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.859 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.859 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.859 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.859 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.859 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.860 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.860 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.860 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.860 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.860 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.860 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.860 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-04-25T11:53:45.861 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-04-25T11:53:45.861 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.861 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.861 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.861 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-04-25T11:53:45.861 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.861 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.861 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.862 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T11:53:45.862 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.862 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.862 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.862 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-04-25T11:53:45.862 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-04-25T11:53:45.862 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.863 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.863 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.863 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.863 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.863 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.863 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.863 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-04-25T11:53:45.863 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.863 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.864 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.864 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-04-25T11:53:45.864 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-04-25T11:53:45.864 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.864 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.864 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.864 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-25T11:53:45.865 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.865 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.865 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.865 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T11:53:45.865 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.865 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.865 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.865 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_no_wait": { 2024-04-25T11:53:45.866 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T11:53:45.866 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.866 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.866 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.866 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.866 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.866 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.866 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.867 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T11:53:45.867 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.867 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.867 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.867 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.867 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.867 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.868 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-04-25T11:53:45.868 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.868 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.868 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.868 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-04-25T11:53:45.868 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-04-25T11:53:45.868 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-25T11:53:45.868 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.868 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.869 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-04-25T11:53:45.869 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.869 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.869 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.869 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.869 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.869 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.869 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.870 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-04-25T11:53:45.870 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-04-25T11:53:45.870 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.870 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.870 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.870 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.870 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.870 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.871 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.871 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.871 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.871 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.871 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.871 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-25T11:53:45.871 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-25T11:53:45.872 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.872 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.872 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.872 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-25T11:53:45.872 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.872 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.872 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.872 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.872 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.873 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.873 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.873 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.873 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.873 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.873 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.873 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.873 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.874 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.874 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.874 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.874 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.874 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.874 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.874 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.874 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.875 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.875 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.875 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.875 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.875 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.875 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.875 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.876 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.876 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.876 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.876 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.876 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.876 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.876 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.876 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.876 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.877 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.877 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.877 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.877 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.877 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.877 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.877 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.878 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.878 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.878 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.878 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.878 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.878 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.882 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.882 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.882 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.882 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.882 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.882 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.882 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.883 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.883 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.883 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.883 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.883 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.883 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.883 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.884 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.884 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.884 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.884 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.884 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.884 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.884 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.884 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.884 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.885 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.885 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-04-25T11:53:45.885 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-04-25T11:53:45.885 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.885 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.885 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.886 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.886 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.886 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.886 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.886 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.886 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.886 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.886 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.887 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-04-25T11:53:45.887 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-04-25T11:53:45.887 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.887 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.887 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.887 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-04-25T11:53:45.887 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.887 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.888 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.888 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.888 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.888 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.888 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.888 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-04-25T11:53:45.888 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-04-25T11:53:45.889 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.889 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.889 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.889 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T11:53:45.889 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.889 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.889 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.889 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.890 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.890 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.890 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.890 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.890 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.890 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.890 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T11:53:45.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.891 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 24893, 2024-04-25T11:53:45.891 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "y", 2024-04-25T11:53:45.891 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_features": 4540701547738038271, 2024-04-25T11:53:45.891 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-04-25T11:53:45.891 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.891 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-04-25T11:53:45.891 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.891 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.892 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.892 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-25T11:53:45.892 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-25T11:53:45.892 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.892 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.892 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.892 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-25T11:53:45.892 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.892 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.893 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.893 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T11:53:45.893 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.893 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.893 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.893 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.893 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.894 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.894 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.894 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.894 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.894 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.894 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.894 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.894 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.895 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.896 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.898 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.899 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.900 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.901 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-04-25T11:53:45.902 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-04-25T11:53:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.903 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2024-04-25T11:53:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.904 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-04-25T11:53:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-04-25T11:53:45.905 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-04-25T11:53:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.906 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-04-25T11:53:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.907 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-04-25T11:53:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-04-25T11:53:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.908 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-04-25T11:53:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.909 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-04-25T11:53:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-04-25T11:53:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.910 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-04-25T11:53:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-04-25T11:53:45.911 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-04-25T11:53:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-04-25T11:53:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.912 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-04-25T11:53:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-04-25T11:53:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-04-25T11:53:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.913 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-04-25T11:53:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.914 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-04-25T11:53:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-04-25T11:53:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.915 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T11:53:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.916 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-04-25T11:53:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-04-25T11:53:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.917 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-04-25T11:53:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-04-25T11:53:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-04-25T11:53:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-04-25T11:53:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.919 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T11:53:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.920 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-04-25T11:53:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-04-25T11:53:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-04-25T11:53:45.921 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T11:53:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:53:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.922 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-04-25T11:53:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-04-25T11:53:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-25T11:53:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-25T11:53:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-25T11:53:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-25T11:53:45.923 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T11:53:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:53:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-04-25T11:53:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-04-25T11:53:45.924 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-25T11:53:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-04-25T11:53:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-25T11:53:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-04-25T11:53:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.925 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T11:53:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-04-25T11:53:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-04-25T11:53:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.926 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T11:53:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T11:53:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T11:53:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.927 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-04-25T11:53:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-04-25T11:53:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-04-25T11:53:45.928 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-04-25T11:53:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-04-25T11:53:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-04-25T11:53:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T11:53:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.929 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-04-25T11:53:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-04-25T11:53:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-04-25T11:53:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.930 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T11:53:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:53:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-04-25T11:53:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-04-25T11:53:45.931 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-25T11:53:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-25T11:53:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-25T11:53:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-25T11:53:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.932 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T11:53:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:53:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.933 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.934 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.936 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:45.937 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:45.938 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.939 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.940 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-04-25T11:53:45.941 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-04-25T11:53:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-25T11:53:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.942 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T11:53:45.943 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.943 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.943 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-25T11:53:45.943 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-25T11:53:45.943 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.943 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.944 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.944 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-04-25T11:53:45.944 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.944 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.944 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.944 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-04-25T11:53:45.944 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-04-25T11:53:45.944 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-04-25T11:53:45.944 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-04-25T11:53:45.945 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-04-25T11:53:45.945 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.945 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-04-25T11:53:45.945 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.945 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.945 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.945 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-04-25T11:53:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-04-25T11:53:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.946 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T11:53:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-25T11:53:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-25T11:53:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.947 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-25T11:53:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T11:53:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.948 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-04-25T11:53:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-04-25T11:53:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-25T11:53:45.949 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-25T11:53:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T11:53:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T11:53:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-04-25T11:53:45.950 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-04-25T11:53:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-25T11:53:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.951 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T11:53:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-04-25T11:53:45.952 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-04-25T11:53:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-04-25T11:53:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.953 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-25T11:53:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T11:53:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-04-25T11:53:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-04-25T11:53:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-04-25T11:53:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.955 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T11:53:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-04-25T11:53:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-04-25T11:53:45.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-04-25T11:53:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T11:53:45.957 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-04-25T11:53:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-04-25T11:53:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.958 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T11:53:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T11:53:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-04-25T11:53:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-04-25T11:53:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-25T11:53:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.960 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T11:53:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-04-25T11:53:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T11:53:45.961 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-04-25T11:53:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T11:53:45.962 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-04-25T11:53:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-04-25T11:53:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.963 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-04-25T11:53:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-04-25T11:53:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-04-25T11:53:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T11:53:45.964 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-04-25T11:53:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-04-25T11:53:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.965 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T11:53:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-04-25T11:53:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-04-25T11:53:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.967 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T11:53:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-04-25T11:53:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-04-25T11:53:45.968 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T11:53:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.969 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-04-25T11:53:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-04-25T11:53:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.970 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T11:53:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-25T11:53:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.971 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-04-25T11:53:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-04-25T11:53:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T11:53:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.972 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-04-25T11:53:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-04-25T11:53:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-04-25T11:53:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.973 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T11:53:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-04-25T11:53:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.974 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-04-25T11:53:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-04-25T11:53:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.975 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T11:53:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-25T11:53:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.976 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-04-25T11:53:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-04-25T11:53:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T11:53:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.977 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-04-25T11:53:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-04-25T11:53:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-04-25T11:53:45.978 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.979 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.979 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.979 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T11:53:45.979 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.979 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.979 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.979 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-04-25T11:53:45.979 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.980 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.980 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.980 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-04-25T11:53:45.980 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-04-25T11:53:45.980 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.980 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.980 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.980 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T11:53:45.981 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.981 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.981 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.981 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-04-25T11:53:45.981 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.981 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.981 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.982 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-04-25T11:53:45.982 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-04-25T11:53:45.982 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.982 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.982 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.982 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T11:53:45.982 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.982 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.983 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.983 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-04-25T11:53:45.983 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.983 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.983 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.983 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-04-25T11:53:45.983 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-04-25T11:53:45.983 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.984 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.984 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.984 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T11:53:45.984 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.984 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.984 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.984 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-04-25T11:53:45.984 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.985 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.985 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.985 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-04-25T11:53:45.985 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-04-25T11:53:45.985 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.985 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.985 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.986 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T11:53:45.986 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.986 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.986 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.986 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-04-25T11:53:45.986 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.986 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.986 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.987 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-04-25T11:53:45.987 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-04-25T11:53:45.987 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.987 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.987 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.987 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T11:53:45.987 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.987 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.988 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.988 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-25T11:53:45.988 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.988 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.988 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.988 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-04-25T11:53:45.988 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-04-25T11:53:45.989 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.989 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.989 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.989 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T11:53:45.989 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.989 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.989 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.989 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-04-25T11:53:45.990 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.990 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.990 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.990 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-04-25T11:53:45.990 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-04-25T11:53:45.990 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.990 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.991 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.991 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T11:53:45.991 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.991 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.991 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.991 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-25T11:53:45.991 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.991 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.991 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.992 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-04-25T11:53:45.992 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-04-25T11:53:45.992 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.992 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.992 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.992 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T11:53:45.992 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.993 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.993 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.993 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-04-25T11:53:45.993 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.993 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.993 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.993 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba": { 2024-04-25T11:53:45.993 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba", 2024-04-25T11:53:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T11:53:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.994 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB container image", 2024-04-25T11:53:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-04-25T11:53:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-04-25T11:53:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:45.995 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T11:53:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-04-25T11:53:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.996 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-04-25T11:53:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-04-25T11:53:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:45.997 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T11:53:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-04-25T11:53:45.998 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-04-25T11:53:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-25T11:53:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:45.999 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T11:53:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-04-25T11:53:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T11:53:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.000 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-04-25T11:53:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T11:53:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.001 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-04-25T11:53:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-04-25T11:53:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-04-25T11:53:46.002 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T11:53:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-04-25T11:53:46.003 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-04-25T11:53:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-04-25T11:53:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.004 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-04-25T11:53:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-04-25T11:53:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-04-25T11:53:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.005 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T11:53:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.006 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-04-25T11:53:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-04-25T11:53:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-25T11:53:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.007 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-04-25T11:53:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: "grafana_dashboards_path": { 2024-04-25T11:53:46.008 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "grafana_dashboards_path", 2024-04-25T11:53:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T11:53:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T11:53:46.009 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.010 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.010 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.010 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-04-25T11:53:46.010 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-04-25T11:53:46.010 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.010 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.010 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-25T11:53:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-04-25T11:53:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.011 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-04-25T11:53:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-04-25T11:53:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.012 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T11:53:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-04-25T11:53:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-04-25T11:53:46.013 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.014 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T11:53:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.015 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.016 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-04-25T11:53:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-04-25T11:53:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.018 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T11:53:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.020 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T11:53:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.021 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.022 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.023 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.024 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.025 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T11:53:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T11:53:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.026 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T11:53:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T11:53:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T11:53:46.027 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-04-25T11:53:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T11:53:46.028 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-04-25T11:53:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-04-25T11:53:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.029 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-04-25T11:53:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-04-25T11:53:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-04-25T11:53:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.031 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T11:53:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-04-25T11:53:46.032 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-04-25T11:53:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-04-25T11:53:46.033 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-25T11:53:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-25T11:53:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.034 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-04-25T11:53:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-04-25T11:53:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-04-25T11:53:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.035 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T11:53:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-04-25T11:53:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-04-25T11:53:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.036 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.037 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.037 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-04-25T11:53:46.037 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.037 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.037 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.037 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T11:53:46.037 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.037 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.038 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.038 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-04-25T11:53:46.038 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-04-25T11:53:46.038 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.038 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.038 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.038 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T11:53:46.038 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.039 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.039 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.039 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T11:53:46.039 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.039 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.039 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.039 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-04-25T11:53:46.040 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-04-25T11:53:46.040 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.040 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.040 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.040 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.040 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.040 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.040 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.040 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T11:53:46.041 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.041 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.041 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.041 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-04-25T11:53:46.041 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-04-25T11:53:46.041 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.041 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.042 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.042 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.042 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.042 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.042 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.042 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T11:53:46.042 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.042 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.042 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.043 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.043 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-04-25T11:53:46.043 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-04-25T11:53:46.043 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.043 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.043 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.043 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.043 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.044 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.044 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.044 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T11:53:46.044 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.044 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.044 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.044 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-04-25T11:53:46.045 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-04-25T11:53:46.045 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.045 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.045 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.045 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.045 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.045 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.045 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.045 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T11:53:46.046 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.046 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.046 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.046 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-04-25T11:53:46.046 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-04-25T11:53:46.046 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.046 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.047 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.047 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.047 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.047 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.047 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.047 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T11:53:46.047 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.047 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.047 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.048 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-04-25T11:53:46.048 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-04-25T11:53:46.048 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.048 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.048 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.048 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-04-25T11:53:46.048 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.049 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.049 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.049 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-04-25T11:53:46.049 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.049 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.049 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.049 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-04-25T11:53:46.049 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-04-25T11:53:46.050 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.050 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.050 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.050 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.050 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.050 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.050 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.050 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T11:53:46.051 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.051 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.051 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.051 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-04-25T11:53:46.051 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-04-25T11:53:46.051 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.051 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.051 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.052 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.052 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.052 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.052 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.052 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T11:53:46.052 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.052 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.052 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.053 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-04-25T11:53:46.053 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-04-25T11:53:46.053 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.053 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.053 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.053 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.053 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.053 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.054 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.054 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T11:53:46.054 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.054 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.054 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.054 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-04-25T11:53:46.054 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-04-25T11:53:46.055 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.055 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.055 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.055 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.055 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.055 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.055 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.055 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T11:53:46.056 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.056 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.056 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.056 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-04-25T11:53:46.056 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-04-25T11:53:46.056 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.056 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.056 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.057 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.057 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.057 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.057 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.057 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T11:53:46.057 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.057 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.058 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.058 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-04-25T11:53:46.058 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-04-25T11:53:46.058 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.058 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.058 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.058 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.058 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.059 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.059 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.059 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T11:53:46.059 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.059 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.059 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.059 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.059 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.060 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.060 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-04-25T11:53:46.060 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.060 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.060 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.060 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.060 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.061 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.061 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.061 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.061 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.061 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.061 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.061 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.061 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.061 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.062 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.062 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.062 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.062 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.062 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.062 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.062 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.062 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.063 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.063 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.063 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.063 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.063 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.063 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.063 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.063 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.064 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.064 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.064 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.064 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.064 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.064 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.064 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.065 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.065 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.065 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.065 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.065 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.065 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.065 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.065 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.065 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.066 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.066 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.066 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.066 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.066 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.066 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.066 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.066 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.067 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.067 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.067 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.067 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.067 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.067 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.067 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.067 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.068 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.068 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.068 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.068 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.068 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.068 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.068 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.069 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-04-25T11:53:46.069 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-04-25T11:53:46.069 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.069 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.069 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.069 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-04-25T11:53:46.069 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.069 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.070 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.070 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T11:53:46.070 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.070 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.070 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.070 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-04-25T11:53:46.070 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-04-25T11:53:46.070 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.071 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.071 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.071 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-04-25T11:53:46.071 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.071 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.071 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.071 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T11:53:46.071 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.072 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.072 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.072 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.072 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.072 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.072 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-04-25T11:53:46.072 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.072 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.073 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.073 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T11:53:46.073 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T11:53:46.073 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.073 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.073 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.073 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:46.073 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.074 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.074 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.074 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.074 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.074 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.074 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.074 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T11:53:46.075 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T11:53:46.075 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.075 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.075 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.075 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.075 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.075 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.075 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.076 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.076 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.076 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.076 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.076 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T11:53:46.076 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T11:53:46.076 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.076 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.077 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.077 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.077 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.077 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.077 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.077 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.077 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.077 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.078 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.078 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-04-25T11:53:46.078 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T11:53:46.078 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.078 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.078 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.078 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.079 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.079 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.079 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.079 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.079 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.079 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.079 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.079 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.079 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T11:53:46.080 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T11:53:46.080 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.080 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.080 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.080 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.080 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.080 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.080 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.081 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.081 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.081 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.081 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.081 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T11:53:46.081 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T11:53:46.081 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.082 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.082 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.082 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.082 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.082 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.082 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.082 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.082 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.082 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.083 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.083 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T11:53:46.083 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T11:53:46.086 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.086 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.086 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.086 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.086 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.086 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.087 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.087 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.087 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.087 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.087 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.087 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T11:53:46.087 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T11:53:46.088 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.088 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.088 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.088 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.088 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.088 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.088 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.088 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.089 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.089 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.089 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.089 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T11:53:46.089 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T11:53:46.089 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.089 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.089 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.090 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.090 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.090 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.090 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.090 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.090 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.090 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.090 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.091 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T11:53:46.091 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T11:53:46.091 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.091 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.091 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.091 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.091 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.091 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.092 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.092 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.092 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.092 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.092 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.092 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T11:53:46.092 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T11:53:46.093 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.093 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.093 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.093 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.093 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.093 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.093 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.093 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.093 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.094 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.094 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.094 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T11:53:46.094 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T11:53:46.094 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.094 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.094 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.095 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.095 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.095 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.095 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.095 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.095 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.095 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.095 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.096 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T11:53:46.096 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T11:53:46.096 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.096 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.096 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.096 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.096 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.096 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.097 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.097 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.097 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.097 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.097 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.097 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T11:53:46.097 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T11:53:46.097 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.098 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.098 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.098 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.098 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.098 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.098 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.098 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.098 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.099 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.099 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.099 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T11:53:46.099 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T11:53:46.099 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.099 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.099 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.100 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-25T11:53:46.100 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.100 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.100 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.100 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.100 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.100 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.100 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.101 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T11:53:46.101 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T11:53:46.101 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.101 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.101 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.101 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.101 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.101 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.102 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.102 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.102 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.102 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.102 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.102 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-04-25T11:53:46.102 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-04-25T11:53:46.102 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.103 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.103 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.103 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.103 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.103 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.103 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.103 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.103 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.104 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.104 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.104 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T11:53:46.104 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T11:53:46.104 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.104 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.104 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.105 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-25T11:53:46.105 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.105 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.105 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.105 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.105 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.105 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.105 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.105 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.106 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T11:53:46.106 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T11:53:46.106 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.106 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.106 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.106 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.106 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.106 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.107 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.107 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.107 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.107 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.107 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.107 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T11:53:46.107 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T11:53:46.107 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.108 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.108 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.108 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.108 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.108 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.108 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.108 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.108 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.109 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.109 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.109 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.109 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T11:53:46.109 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T11:53:46.109 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.109 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.109 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.110 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.110 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.110 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.110 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.110 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.110 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.110 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.110 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.111 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T11:53:46.111 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T11:53:46.111 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.111 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.111 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.111 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.111 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.111 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.112 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.112 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.112 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.112 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.112 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.112 INFO:teuthology.orchestra.run.smithi083.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T11:53:46.112 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T11:53:46.112 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.113 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.113 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.113 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "[]", 2024-04-25T11:53:46.113 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.113 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.113 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.113 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.113 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.114 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.114 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.114 INFO:teuthology.orchestra.run.smithi083.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T11:53:46.114 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T11:53:46.114 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.114 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.114 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.115 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "{}", 2024-04-25T11:53:46.115 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.115 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.115 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.115 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.115 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.115 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.115 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.116 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T11:53:46.116 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T11:53:46.116 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.116 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.116 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.116 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.116 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.116 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.116 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.117 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.117 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.117 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.117 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.117 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T11:53:46.117 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T11:53:46.117 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.118 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.118 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.118 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.118 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.118 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.118 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.118 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.118 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.119 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.119 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.119 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T11:53:46.119 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T11:53:46.119 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.119 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.119 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.119 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.120 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.120 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.120 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.120 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.120 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.120 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.120 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.121 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T11:53:46.121 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T11:53:46.121 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.121 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.121 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.121 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.121 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.121 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.122 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.122 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.122 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.122 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.122 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.122 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T11:53:46.122 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T11:53:46.123 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.123 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.123 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.123 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.123 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.123 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.123 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.123 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.123 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.124 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.124 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.124 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T11:53:46.124 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T11:53:46.124 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.124 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.124 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.125 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.125 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.125 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.125 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.125 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.125 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.125 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.125 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.126 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T11:53:46.126 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T11:53:46.126 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.126 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.126 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.126 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.126 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.126 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.127 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.127 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.127 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.127 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.127 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.127 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.127 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T11:53:46.127 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T11:53:46.128 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.128 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.128 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.128 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.128 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.128 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.128 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.128 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.128 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.129 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.129 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.129 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T11:53:46.129 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T11:53:46.129 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.129 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.129 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.130 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.130 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.130 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.130 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.130 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.130 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.130 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.130 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.131 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T11:53:46.131 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T11:53:46.131 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.131 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.131 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.131 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.131 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.131 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.132 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.132 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.132 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.132 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.132 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.132 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T11:53:46.132 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T11:53:46.133 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.133 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.133 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.133 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T11:53:46.133 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.133 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.133 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.133 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.133 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.134 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.134 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.134 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T11:53:46.134 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T11:53:46.134 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.134 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.134 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.135 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:46.135 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.135 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.135 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.135 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.135 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.135 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.135 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.136 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T11:53:46.136 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T11:53:46.136 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.136 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.136 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.136 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-04-25T11:53:46.136 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.136 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.137 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.137 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.137 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.137 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.137 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.137 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T11:53:46.137 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T11:53:46.137 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.138 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.138 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.138 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-04-25T11:53:46.138 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.138 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.138 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.138 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.138 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.139 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.139 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.139 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T11:53:46.139 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T11:53:46.139 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.139 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.139 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.139 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.140 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.140 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.140 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.140 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.140 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.140 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.140 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.141 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T11:53:46.141 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T11:53:46.141 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.141 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.141 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.141 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-25T11:53:46.141 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.141 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.141 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.142 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.142 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.142 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.142 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.142 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T11:53:46.142 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T11:53:46.142 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.143 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.143 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.143 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.143 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.143 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.143 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.143 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.143 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.143 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.144 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.144 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.144 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T11:53:46.144 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T11:53:46.144 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.144 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.144 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.144 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.145 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.145 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.145 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.145 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.145 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.145 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.145 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.145 INFO:teuthology.orchestra.run.smithi083.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T11:53:46.146 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T11:53:46.146 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.146 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.146 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.146 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.146 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.146 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.146 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.147 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.147 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.147 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.147 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.147 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T11:53:46.147 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T11:53:46.147 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.148 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.148 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.148 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-25T11:53:46.148 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.148 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.148 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.148 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.148 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.149 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.149 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.149 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T11:53:46.149 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T11:53:46.149 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.149 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.149 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.149 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:46.150 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.150 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.150 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.150 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.150 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.150 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.150 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.151 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T11:53:46.151 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T11:53:46.151 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.151 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.151 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.151 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-25T11:53:46.151 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.151 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.151 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.152 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.152 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.152 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.152 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.152 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-04-25T11:53:46.152 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-04-25T11:53:46.152 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.153 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.153 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.153 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.153 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.153 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.153 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.153 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.153 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.153 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.154 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.154 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-04-25T11:53:46.154 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-04-25T11:53:46.154 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.154 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.154 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.154 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.155 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.155 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.155 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.155 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.155 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.155 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.155 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.155 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-04-25T11:53:46.156 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-04-25T11:53:46.156 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.156 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.156 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.156 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.156 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.156 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.156 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.157 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-04-25T11:53:46.157 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.157 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.157 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.157 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-04-25T11:53:46.157 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-04-25T11:53:46.157 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.157 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.158 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.158 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-04-25T11:53:46.158 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.158 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.158 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.158 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.158 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.158 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.159 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.159 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-25T11:53:46.159 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-25T11:53:46.159 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.159 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.159 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.159 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.160 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.160 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.160 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.160 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.160 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.160 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.160 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.160 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.161 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.161 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.161 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.161 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.161 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.161 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.161 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.161 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.162 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.162 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.162 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.162 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.162 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.162 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.162 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.162 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.162 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.163 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.163 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.163 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.163 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.163 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.163 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.163 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.164 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.164 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.164 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.164 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.164 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.164 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.164 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.164 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.165 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.165 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.165 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.165 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.165 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.165 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.165 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.165 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.165 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.166 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.166 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.166 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.166 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.166 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.166 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.166 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.166 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.167 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.167 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.167 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.167 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.167 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.167 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.167 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.167 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.168 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.168 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.168 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.168 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.168 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.168 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.168 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.168 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.169 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-04-25T11:53:46.169 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-04-25T11:53:46.169 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.169 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.169 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.169 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.169 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.169 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.170 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.170 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-04-25T11:53:46.170 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.170 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.170 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.170 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-04-25T11:53:46.170 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T11:53:46.171 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.171 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.171 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.171 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.171 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.171 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.171 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.171 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.172 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.172 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.172 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.172 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-25T11:53:46.172 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-25T11:53:46.172 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.172 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.172 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.173 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-25T11:53:46.173 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.173 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.173 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.173 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.173 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.173 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.173 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.174 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-25T11:53:46.174 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-25T11:53:46.174 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.174 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.174 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.174 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-04-25T11:53:46.174 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.174 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.175 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.175 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.175 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.175 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.175 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.175 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-25T11:53:46.175 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-25T11:53:46.176 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.176 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.176 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.176 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.176 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.176 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.176 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.176 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.176 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.177 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.177 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.177 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-04-25T11:53:46.177 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-04-25T11:53:46.177 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.177 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.177 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.178 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-04-25T11:53:46.178 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.178 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.178 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.178 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.178 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.178 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.178 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.179 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-25T11:53:46.179 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-25T11:53:46.179 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.179 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.179 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.179 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-04-25T11:53:46.179 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.179 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.180 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.180 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.180 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-04-25T11:53:46.180 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.180 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.180 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.180 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.180 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-25T11:53:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-25T11:53:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-25T11:53:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-25T11:53:46.181 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-25T11:53:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-04-25T11:53:46.182 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-04-25T11:53:46.183 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.183 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.183 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.183 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.183 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.183 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.183 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.183 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.183 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.184 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.184 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.184 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.184 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.184 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-04-25T11:53:46.184 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.185 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.185 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.185 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-04-25T11:53:46.185 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-04-25T11:53:46.185 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.185 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.185 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.185 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.185 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.186 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.186 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.186 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-04-25T11:53:46.186 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.186 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.186 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.186 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.187 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.187 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.187 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.187 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.187 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.187 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.187 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.187 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.188 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.188 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.188 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.188 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.188 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.188 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.188 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.188 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.189 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.189 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.189 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.189 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.189 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.189 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.189 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.190 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.190 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.190 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.190 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.190 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.190 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.190 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.190 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.191 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.191 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.191 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.191 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.191 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.191 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.191 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.191 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.191 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.192 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.192 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.192 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.192 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.192 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.192 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.192 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.192 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.193 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.193 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.193 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.193 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.193 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.193 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.193 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.194 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.194 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.194 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.194 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.194 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.194 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.194 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.194 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.195 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.195 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.195 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.195 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-04-25T11:53:46.195 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-04-25T11:53:46.195 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.195 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.196 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.196 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-04-25T11:53:46.196 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.196 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.196 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.196 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T11:53:46.196 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.196 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.197 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.197 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-04-25T11:53:46.197 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-04-25T11:53:46.197 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.197 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.197 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.197 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-04-25T11:53:46.197 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.198 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.198 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.198 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T11:53:46.198 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.198 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.198 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.198 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-04-25T11:53:46.199 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-04-25T11:53:46.199 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.199 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.199 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.199 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-04-25T11:53:46.199 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.199 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.199 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.200 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-04-25T11:53:46.200 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.200 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.200 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.200 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-04-25T11:53:46.200 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-04-25T11:53:46.200 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.200 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.201 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.201 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-25T11:53:46.201 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.201 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.201 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.201 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T11:53:46.201 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.201 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.202 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.202 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.202 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-04-25T11:53:46.202 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-04-25T11:53:46.202 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.202 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.202 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.202 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.202 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.203 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.203 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.203 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T11:53:46.203 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.203 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.203 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.203 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-25T11:53:46.204 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-25T11:53:46.204 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.204 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.204 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.204 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-25T11:53:46.204 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.204 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.204 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.205 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T11:53:46.205 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.205 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.205 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.205 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-04-25T11:53:46.205 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-04-25T11:53:46.205 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.205 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.206 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.206 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-04-25T11:53:46.206 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.206 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.206 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.206 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T11:53:46.206 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.206 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.210 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.210 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.210 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.210 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.210 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.210 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-04-25T11:53:46.210 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.211 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.211 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.211 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.211 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.211 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.211 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.211 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.211 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.212 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.212 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.212 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.212 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.212 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.212 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.212 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.212 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.212 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.213 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.213 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.213 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.213 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.213 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.213 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.213 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.213 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.214 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.214 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.214 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.214 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.214 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.214 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.214 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.215 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.215 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.215 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.215 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.215 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.215 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.216 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.216 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.216 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.216 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.216 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.216 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.216 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.216 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.217 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.217 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.217 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.217 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.217 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.217 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.217 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.217 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.218 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.218 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.218 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.218 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.218 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.218 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.218 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.219 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.219 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.219 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.219 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.219 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.219 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.219 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-04-25T11:53:46.220 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-04-25T11:53:46.220 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.220 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.220 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.220 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-25T11:53:46.220 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.220 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.220 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.221 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.221 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.221 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.221 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.221 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-04-25T11:53:46.221 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-04-25T11:53:46.221 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.222 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.222 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.222 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-04-25T11:53:46.222 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.222 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.222 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.222 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.223 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.223 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.223 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.223 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-25T11:53:46.223 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-25T11:53:46.223 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.223 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.223 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.224 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-25T11:53:46.224 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.224 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.224 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.224 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.224 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.224 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.224 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.224 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.225 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.225 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.225 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-04-25T11:53:46.225 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-04-25T11:53:46.225 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-04-25T11:53:46.225 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.225 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-04-25T11:53:46.226 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-04-25T11:53:46.226 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.226 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.226 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.226 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-04-25T11:53:46.226 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.226 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.226 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.227 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T11:53:46.227 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.227 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.227 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.227 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-04-25T11:53:46.227 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-04-25T11:53:46.227 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.227 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.228 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.228 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-04-25T11:53:46.228 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.228 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.228 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.228 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T11:53:46.228 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.228 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.229 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.229 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-04-25T11:53:46.229 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-04-25T11:53:46.229 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.229 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.229 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.229 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.229 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.230 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.230 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.230 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-04-25T11:53:46.230 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.230 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.230 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.230 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.230 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-25T11:53:46.231 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-25T11:53:46.231 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.231 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.231 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.231 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-04-25T11:53:46.231 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-04-25T11:53:46.231 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.231 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.232 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T11:53:46.232 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.232 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.232 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.232 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.232 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.232 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.232 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.233 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.233 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.233 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.233 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.233 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.233 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.233 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.233 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.233 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.234 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.234 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.234 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.234 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.234 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.234 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.234 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.235 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.235 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.235 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.235 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.235 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.235 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.235 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.235 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.236 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.236 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.236 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.236 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.236 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.236 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.236 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.236 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.237 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.237 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.237 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.237 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.237 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.237 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.237 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.237 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.237 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.238 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.238 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.238 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.238 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.238 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.238 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.238 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.238 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.239 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.239 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.239 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.239 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.239 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.239 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.239 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.240 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.240 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.240 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.240 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.240 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.240 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.240 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-04-25T11:53:46.240 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-04-25T11:53:46.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:45.158+0000 7f5286d76640 1 -- 172.21.15.83:0/1807452094 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f526c0a3d20 msgr2=0x7f526c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:46.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:45.158+0000 7f5286d76640 1 --2- 172.21.15.83:0/1807452094 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f526c0a3d20 0x7f526c0a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5280077640 tx=0x7f527800a590 comp rx=0 tx=0).stop 2024-04-25T11:53:46.242 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.242 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.242 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.242 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.242 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.242 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.243 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.243 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-04-25T11:53:46.243 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.243 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.243 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.243 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-04-25T11:53:46.243 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-04-25T11:53:46.243 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.244 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.244 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.244 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-04-25T11:53:46.244 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.244 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.244 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.244 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-04-25T11:53:46.244 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.245 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.245 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.245 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-25T11:53:46.245 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-25T11:53:46.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.245 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.245 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.245 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-04-25T11:53:46.246 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.246 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.246 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.246 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:53:46.246 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.246 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.246 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.246 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.247 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-04-25T11:53:46.247 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-04-25T11:53:46.247 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.247 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.247 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.247 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-25T11:53:46.247 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-25T11:53:46.247 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-04-25T11:53:46.248 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.248 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T11:53:46.248 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.248 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.248 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.248 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-04-25T11:53:46.248 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-04-25T11:53:46.249 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.249 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.249 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.249 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.249 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.249 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.249 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.249 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-04-25T11:53:46.249 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.250 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.250 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.250 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.250 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-04-25T11:53:46.250 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-04-25T11:53:46.250 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.250 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.250 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.251 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-04-25T11:53:46.251 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.251 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.251 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.251 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:53:46.251 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.251 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.251 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.251 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.252 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.252 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-04-25T11:53:46.252 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.252 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.252 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.252 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.253 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.253 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.253 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.253 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.253 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.253 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.253 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.253 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.254 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.254 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.254 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.254 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.254 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.254 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.254 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.254 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.255 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.255 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.255 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.255 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.255 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.255 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.255 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.256 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.256 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.256 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.256 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.256 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.256 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.256 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.256 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.257 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.257 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.257 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.257 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.257 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.257 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.257 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.257 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.258 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.258 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.258 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.258 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.258 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.258 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.258 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.258 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.259 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.259 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.259 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.259 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.259 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.259 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.259 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.259 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.260 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.260 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.260 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.260 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.260 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.260 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.260 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.260 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.261 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.261 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.261 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.261 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-04-25T11:53:46.261 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.261 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.261 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.261 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.262 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.262 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.262 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.262 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.262 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.262 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.262 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.263 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.263 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.263 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.263 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.263 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.263 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.263 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.263 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.264 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.264 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.264 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.264 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.264 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.264 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.264 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.264 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.265 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.265 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.265 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.265 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.265 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.265 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.265 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.265 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.266 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.266 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.266 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.266 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.266 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.266 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.266 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.267 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.267 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.267 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.267 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.267 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.267 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.267 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.267 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.267 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.268 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.268 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.268 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.268 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.268 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.268 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.268 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.269 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.269 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.269 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.269 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.269 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.269 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.269 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.269 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.269 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.270 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.270 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.270 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.270 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.270 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-04-25T11:53:46.270 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.270 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.271 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.271 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-04-25T11:53:46.271 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-04-25T11:53:46.271 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.271 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.271 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.271 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-04-25T11:53:46.271 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.271 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.272 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.272 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T11:53:46.272 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.272 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.272 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.272 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-04-25T11:53:46.272 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-04-25T11:53:46.273 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.273 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.273 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.273 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:46.273 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-04-25T11:53:46.273 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.273 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.273 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T11:53:46.273 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.274 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.274 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.274 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.274 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.274 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.274 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.274 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.275 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.275 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.275 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.275 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.275 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.275 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.275 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.275 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.275 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.276 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.276 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.276 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.276 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.276 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.276 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.276 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.277 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.277 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.277 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.277 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.277 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.277 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.277 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.278 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.278 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.278 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.278 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.278 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.278 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.278 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.279 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.279 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.279 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.279 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.279 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.279 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.279 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.279 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.279 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.280 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.280 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.280 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.280 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.280 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.280 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.280 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.281 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.281 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.281 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.281 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.281 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.281 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.281 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.281 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.281 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.282 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.282 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.282 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.282 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.282 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.282 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.283 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-04-25T11:53:46.283 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.283 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.283 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.283 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-04-25T11:53:46.283 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-04-25T11:53:46.283 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.283 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.283 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.284 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-04-25T11:53:46.284 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.284 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.284 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.284 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-04-25T11:53:46.284 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T11:53:46.284 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.284 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.285 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.285 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.285 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.285 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.285 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.285 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.285 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.285 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.286 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.286 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.286 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.286 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.286 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.286 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.286 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.286 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.287 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.287 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.287 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.287 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.287 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.287 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.288 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.288 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.288 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.288 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.288 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.288 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.288 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.288 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.289 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.289 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.289 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.289 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.289 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.289 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.289 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.290 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.290 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.290 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.290 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.290 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.290 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.290 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.290 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.290 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.291 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.291 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.291 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.291 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.291 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.291 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.291 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.292 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.292 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.292 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.292 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.292 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.292 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.292 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.292 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.292 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.293 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.293 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.293 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.293 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.293 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-04-25T11:53:46.293 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-04-25T11:53:46.293 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.294 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.294 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.294 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.294 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.294 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.294 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-04-25T11:53:46.294 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T11:53:46.294 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.295 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.295 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-04-25T11:53:46.295 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-04-25T11:53:46.295 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.295 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.295 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.295 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-04-25T11:53:46.296 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.296 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.296 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.296 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-04-25T11:53:46.296 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.296 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.296 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.297 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-04-25T11:53:46.297 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-04-25T11:53:46.297 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.297 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.297 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.297 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-04-25T11:53:46.297 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.297 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.297 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.298 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T11:53:46.298 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.298 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.298 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.298 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-04-25T11:53:46.298 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-04-25T11:53:46.298 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.299 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.299 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.299 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.299 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.299 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.299 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.299 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-04-25T11:53:46.299 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.299 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-04-25T11:53:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-04-25T11:53:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.300 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-04-25T11:53:46.301 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.301 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.301 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.301 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T11:53:46.301 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T11:53:46.301 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.301 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.302 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-04-25T11:53:46.302 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.302 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.302 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.302 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.302 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.303 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.303 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.303 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.303 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.303 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.303 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.303 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.303 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.303 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.304 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.304 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.304 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.304 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.304 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.304 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.304 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.304 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.305 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.305 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.305 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.305 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.305 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.305 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.305 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.305 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.306 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.306 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.306 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.306 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.306 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.306 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.306 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.307 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.307 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.307 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.307 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.307 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.307 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.307 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.307 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.308 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.308 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.308 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.308 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.308 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.308 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.308 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.308 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.308 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.309 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.309 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.309 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.309 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.310 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.310 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.310 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.310 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.310 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.310 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.310 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.310 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.311 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.311 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.311 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.311 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-04-25T11:53:46.311 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.311 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.311 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.311 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.312 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.312 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.312 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.312 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.312 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.312 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.312 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.312 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.313 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.313 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.313 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.313 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.313 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.313 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.313 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.313 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.313 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.314 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.314 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.314 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.314 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.314 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.314 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.314 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.315 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.315 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.315 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.315 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.315 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.315 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.315 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.315 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.316 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.316 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.316 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.316 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.316 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.316 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.316 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.316 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.317 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.317 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.317 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.317 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.317 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.317 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.317 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.317 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.317 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.318 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.318 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.318 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.318 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.318 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.318 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.318 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.319 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.319 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.319 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.319 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.319 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.319 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.319 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.319 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.319 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.320 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.320 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-04-25T11:53:46.320 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.320 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.320 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.321 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.321 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.321 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.321 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.321 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.321 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.321 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.321 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.322 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.322 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.322 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.322 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.322 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.322 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.322 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.323 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.323 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.323 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.323 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.323 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.323 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.323 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.324 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.324 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.324 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.324 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.324 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.324 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.324 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.324 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.324 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.325 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.325 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.325 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.325 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.325 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.325 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.325 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.326 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.326 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.326 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.326 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.326 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.326 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.326 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.326 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.327 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.327 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.327 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.327 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.327 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.327 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.327 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.328 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.328 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.328 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.328 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.328 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.328 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.328 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.328 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.329 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.329 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.329 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.329 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.329 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-25T11:53:46.329 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.329 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.330 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.330 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-04-25T11:53:46.330 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-04-25T11:53:46.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.330 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.330 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.330 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.330 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.331 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.331 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.331 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T11:53:46.331 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.331 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.331 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.331 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.331 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.332 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.332 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.332 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.332 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.332 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.332 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.332 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.333 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.333 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.333 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.333 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:45.158+0000 7f5286d76640 1 -- 172.21.15.83:0/1807452094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280105e90 msgr2=0x7f528019cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:53:46.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:45.158+0000 7f5286d76640 1 --2- 172.21.15.83:0/1807452094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280105e90 0x7f528019cf40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5270005940 tx=0x7f52700042f0 comp rx=0 tx=0).stop 2024-04-25T11:53:46.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:45.158+0000 7f5286d76640 1 -- 172.21.15.83:0/1807452094 shutdown_connections 2024-04-25T11:53:46.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:45.158+0000 7f5286d76640 1 --2- 172.21.15.83:0/1807452094 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f526c0a3d20 0x7f526c0a61e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:46.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:45.158+0000 7f5286d76640 1 --2- 172.21.15.83:0/1807452094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5280105e90 0x7f528019cf40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:46.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:45.158+0000 7f5286d76640 1 --2- 172.21.15.83:0/1807452094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5280102370 0x7f5280198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:46.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:45.158+0000 7f5286d76640 1 --2- 172.21.15.83:0/1807452094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52801019b0 0x7f5280198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:53:46.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:45.158+0000 7f5286d76640 1 -- 172.21.15.83:0/1807452094 >> 172.21.15.83:0/1807452094 conn(0x7f5280076130 msgr2=0x7f5280100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:53:46.335 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.335 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.335 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.335 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.335 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.335 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.336 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.336 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.336 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.336 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.336 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.336 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.336 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.336 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.336 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.337 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.337 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.337 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.337 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.337 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.337 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.337 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.338 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.338 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.338 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.338 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.338 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.338 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.338 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.338 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.339 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.339 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.339 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.339 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.339 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.339 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.339 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.339 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.339 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.340 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.340 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.340 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.340 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.340 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.340 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.340 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.340 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.340 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.341 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.341 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.341 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.341 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.341 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.341 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.341 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-04-25T11:53:46.342 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-25T11:53:46.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.342 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.342 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.342 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.342 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.342 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.342 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.343 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-04-25T11:53:46.343 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-04-25T11:53:46.343 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-04-25T11:53:46.343 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.343 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-04-25T11:53:46.343 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.343 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.343 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.343 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.344 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.344 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.344 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-04-25T11:53:46.344 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.344 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.344 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.344 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.345 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.345 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.345 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.345 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.345 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.345 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.345 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.345 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.345 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.346 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.346 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.346 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.346 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.346 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.346 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.346 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.346 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.347 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.347 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.347 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.347 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.347 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.347 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.347 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.348 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.348 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.348 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.348 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.348 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.348 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.348 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.349 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.349 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.349 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.349 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.349 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.349 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.349 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.349 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.349 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.350 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.350 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.350 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.350 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.350 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.350 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.350 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.350 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.351 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.351 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.351 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.351 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.351 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.351 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.351 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.351 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.352 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.352 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.352 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.352 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.352 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.352 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.352 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.352 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.352 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.353 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.353 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.353 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-04-25T11:53:46.353 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.353 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.353 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.353 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.354 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.354 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.354 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.354 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.354 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.354 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.354 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.354 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.355 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.355 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.355 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.355 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.355 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.355 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.355 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.355 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.355 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.356 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.356 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.356 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.356 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.356 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.356 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.356 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.357 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.357 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.357 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.357 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.357 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.357 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.357 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.357 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.358 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.358 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.358 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.358 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.358 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.358 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.358 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.358 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.359 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.359 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.359 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.359 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.359 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.359 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.359 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.359 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.360 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.360 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.360 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.360 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.360 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.360 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.361 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.361 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.361 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.361 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.361 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.361 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.361 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.361 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.361 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.362 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.362 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.362 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.362 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-04-25T11:53:46.362 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.362 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.363 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.363 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.363 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.363 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.363 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.363 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.363 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.363 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.364 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.364 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.364 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.364 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.364 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.364 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.364 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.364 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.365 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.365 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.365 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.365 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.365 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.365 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.365 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.365 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.366 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.366 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.366 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.366 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.366 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.366 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.366 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.366 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.367 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.367 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.367 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.367 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.367 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.367 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.367 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.367 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.368 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.369 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.370 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.370 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.370 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.370 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.370 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.370 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.370 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.370 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.371 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.371 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.371 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.371 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-25T11:53:46.371 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-25T11:53:46.371 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.371 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.371 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.372 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-25T11:53:46.372 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.372 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.372 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.372 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.372 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.372 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.372 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.373 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-04-25T11:53:46.373 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-04-25T11:53:46.373 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:46.373 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.373 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.373 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-25T11:53:46.373 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-04-25T11:53:46.373 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.374 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.374 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-04-25T11:53:46.374 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T11:53:46.374 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.374 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.374 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.374 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.375 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.375 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-04-25T11:53:46.375 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.375 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.375 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.375 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-04-25T11:53:46.375 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-04-25T11:53:46.375 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.375 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.376 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.376 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.376 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.376 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.376 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.376 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T11:53:46.376 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.376 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.377 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.377 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-25T11:53:46.377 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-25T11:53:46.377 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.377 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.377 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.377 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.377 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.378 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.378 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.378 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.378 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.378 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.378 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.378 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.379 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.379 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.379 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.379 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.379 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.379 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.379 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.379 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.380 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.380 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.380 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.380 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.380 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.380 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.380 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.380 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.380 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.381 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.381 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.381 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.381 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.381 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.381 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.381 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.382 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.382 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.382 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.382 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.382 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.382 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.382 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.382 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.383 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.383 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.383 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.383 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.383 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.383 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.383 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.383 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.384 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.384 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.384 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.384 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.384 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.384 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.384 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.384 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.385 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.385 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.385 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.385 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.385 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.385 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.385 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.385 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.386 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.386 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.386 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.386 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.386 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.386 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.386 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.386 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.387 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.387 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-04-25T11:53:46.387 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-04-25T11:53:46.387 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.387 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.387 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.387 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-04-25T11:53:46.387 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.388 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.388 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.388 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-04-25T11:53:46.388 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.388 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.388 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.388 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-25T11:53:46.389 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-25T11:53:46.389 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.389 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.389 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.389 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-25T11:53:46.389 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.389 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.389 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.389 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-04-25T11:53:46.390 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.390 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.390 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.390 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.390 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-04-25T11:53:46.391 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.391 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.391 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.391 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-04-25T11:53:46.391 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-04-25T11:53:46.391 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.391 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.391 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.391 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.392 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.392 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.392 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.392 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.392 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.392 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.392 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.393 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-04-25T11:53:46.393 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-04-25T11:53:46.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.393 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.393 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.393 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.393 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.393 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.393 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.394 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T11:53:46.394 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T11:53:46.394 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.394 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.394 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.394 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.394 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.395 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.395 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.395 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.395 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.395 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.395 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.395 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.395 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.395 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.396 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.396 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.396 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.396 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.396 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.396 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.396 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.396 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.397 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.397 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.397 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.397 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.397 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.397 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.397 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.397 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.398 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.398 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.398 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.398 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.398 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.398 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.398 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.399 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.399 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.399 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.399 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.399 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.399 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.399 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.399 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.399 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.400 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.400 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.400 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.400 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.400 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.400 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.400 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.400 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.401 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.401 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.401 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.401 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.401 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.401 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.401 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.401 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.402 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.402 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.402 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.402 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.402 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.402 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.402 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-04-25T11:53:46.403 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-04-25T11:53:46.403 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.403 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.403 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.403 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.403 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.403 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.403 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.404 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.404 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.404 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.404 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.404 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T11:53:46.404 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T11:53:46.404 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.404 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.405 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.405 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-04-25T11:53:46.405 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.405 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.405 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.405 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.405 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.405 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.406 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.406 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.406 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-04-25T11:53:46.406 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-04-25T11:53:46.406 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:46.406 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.406 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.406 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-04-25T11:53:46.407 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.407 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.407 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.407 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.407 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.407 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.407 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.407 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-25T11:53:46.408 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-25T11:53:46.408 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.408 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.408 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.408 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-25T11:53:46.408 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.408 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.408 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.409 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T11:53:46.409 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.409 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.409 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.409 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-25T11:53:46.409 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-25T11:53:46.409 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.410 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.410 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.410 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-04-25T11:53:46.410 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.410 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.410 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.410 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T11:53:46.410 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.410 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.411 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.411 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-04-25T11:53:46.411 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-04-25T11:53:46.411 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.411 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.411 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.411 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-04-25T11:53:46.412 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.412 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.412 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.412 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.412 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.412 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.412 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.413 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-25T11:53:46.413 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-25T11:53:46.413 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.413 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.413 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.413 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-04-25T11:53:46.413 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.413 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.413 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.414 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-04-25T11:53:46.414 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-04-25T11:53:46.414 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.414 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.414 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.414 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.414 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.415 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-25T11:53:46.415 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-25T11:53:46.415 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.415 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.415 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.415 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-25T11:53:46.415 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-25T11:53:46.415 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-25T11:53:46.416 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.416 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.416 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.416 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.416 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.416 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.416 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.417 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-04-25T11:53:46.417 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.417 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.417 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.417 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.417 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.417 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.417 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.417 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.418 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.418 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.418 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.418 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.418 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.418 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.418 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.418 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.419 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.419 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.419 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.419 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.419 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.419 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.419 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.419 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.420 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.420 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.420 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.420 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.420 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.420 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.420 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.420 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.421 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.421 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.421 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.421 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.421 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.421 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.421 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.421 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.422 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.422 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.422 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.422 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.422 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.422 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.422 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.423 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.423 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.423 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.423 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.423 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.423 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.423 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.423 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.424 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.424 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.424 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.424 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.424 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.424 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.424 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.424 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.425 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.425 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.425 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.425 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.425 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:45.158+0000 7f5286d76640 1 -- 172.21.15.83:0/1807452094 shutdown_connections 2024-04-25T11:53:46.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:53:45.158+0000 7f5286d76640 1 -- 172.21.15.83:0/1807452094 wait complete. 2024-04-25T11:53:46.427 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:44 smithi083 ceph-mon[29330]: from='client.15462 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:53:46.427 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:44 smithi083 ceph-mon[29330]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T11:53:46.427 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:44 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3304799787' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:53:46.427 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:44 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/994890158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:53:46.427 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:44 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2030796926' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:53:46.427 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:44 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/340197396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:53:46.428 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:45 smithi111 ceph-mon[32784]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:46.428 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:45 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1807452094' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T11:53:46.428 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:45 smithi094 ceph-mon[33032]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:46.428 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:45 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1807452094' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T11:53:46.431 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-04-25T11:53:46.431 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-04-25T11:53:46.431 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.431 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.432 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.432 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:46.432 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.432 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.432 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.432 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.432 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.432 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.432 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.433 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-04-25T11:53:46.433 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T11:53:46.433 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.433 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.433 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.433 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.433 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.434 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.434 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.434 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.434 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.434 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.434 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.434 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-04-25T11:53:46.435 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-04-25T11:53:46.435 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.435 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.435 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.435 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.435 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.435 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.435 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.435 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.436 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.436 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.436 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.436 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.436 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.436 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.436 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-04-25T11:53:46.437 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.437 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.437 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.437 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-04-25T11:53:46.437 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-04-25T11:53:46.437 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.437 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.437 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.437 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.438 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.438 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.438 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.438 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.438 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.438 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.438 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.439 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-25T11:53:46.439 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-25T11:53:46.439 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.439 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.439 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.439 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.439 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.439 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.440 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.440 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.440 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.440 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.440 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.440 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.440 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.440 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.440 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.441 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.441 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.441 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.441 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.441 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.441 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.441 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.441 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.442 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.442 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.442 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.442 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.442 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.442 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.442 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.442 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.443 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.443 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.443 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.443 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.443 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.443 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.443 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.444 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.444 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.444 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.444 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.444 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.444 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.444 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.444 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.444 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.445 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.445 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.445 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.445 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.445 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.445 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.445 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.445 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.446 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.446 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.446 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.446 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.446 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.446 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.446 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.446 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.446 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.447 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.447 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.447 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.447 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.447 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.447 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.447 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.448 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.448 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.448 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.448 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.448 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.448 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.448 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.448 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-25T11:53:46.449 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-25T11:53:46.449 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.449 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.449 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.449 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.449 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.449 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.449 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.450 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.450 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.450 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.450 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.450 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-25T11:53:46.450 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-25T11:53:46.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.450 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.451 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.451 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.451 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.451 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.451 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.451 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.451 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.451 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.452 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.452 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.452 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.452 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.452 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-04-25T11:53:46.452 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.452 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.452 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.453 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.453 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.453 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.453 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.453 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.453 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.453 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.453 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.454 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.454 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.454 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.454 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.454 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.454 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.454 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.454 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.455 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.455 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.455 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.455 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.455 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.455 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.455 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.455 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.456 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.456 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.456 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.456 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.456 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.456 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.456 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.456 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.457 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.457 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.457 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.457 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.457 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.457 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.457 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.457 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.458 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.458 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.458 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.458 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.458 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.458 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.458 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.458 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.459 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.459 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.459 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.459 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.459 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.459 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.459 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.459 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.460 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.460 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.460 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.460 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.460 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.460 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.460 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.460 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.461 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.461 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.461 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.461 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.461 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-04-25T11:53:46.461 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.461 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.462 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.462 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.462 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.462 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.462 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.462 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.462 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.462 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.463 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.463 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.463 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.463 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.463 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.463 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.463 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.463 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.464 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.464 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.464 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.464 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.464 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.464 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.464 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.464 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.464 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.465 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.465 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.465 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.465 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.465 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.465 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.465 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.465 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.465 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.466 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.466 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.466 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.466 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.466 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.466 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.466 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.467 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.467 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.467 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.467 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.467 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.467 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.467 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.467 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.467 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.468 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.468 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.468 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.468 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.468 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.468 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.468 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.468 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.469 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.469 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.469 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.469 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.469 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.469 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.469 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.469 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.469 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.470 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.470 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.470 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-04-25T11:53:46.470 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-04-25T11:53:46.470 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.470 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.470 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.471 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-04-25T11:53:46.471 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.471 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.471 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.471 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T11:53:46.471 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.471 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.471 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.472 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.472 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.472 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-04-25T11:53:46.472 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.472 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.472 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.473 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.473 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.473 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.473 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.473 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.473 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.473 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.473 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.474 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.474 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.474 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.474 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.474 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.474 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.474 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.474 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.474 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.475 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.475 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.475 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.475 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.475 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.475 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.475 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.475 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.476 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.476 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.476 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.476 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.476 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.476 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.476 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.476 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.477 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.477 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.477 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.477 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.477 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.477 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.477 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.477 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.478 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.478 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.478 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.478 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.478 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.478 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.478 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.479 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.479 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.479 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.479 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.479 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.479 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.479 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.479 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.480 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.480 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.480 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.480 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.480 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.480 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.480 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.480 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.481 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.481 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.481 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.481 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.481 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-04-25T11:53:46.481 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-04-25T11:53:46.481 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.481 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.482 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.482 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.482 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.482 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.482 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.482 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.482 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.482 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.483 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.483 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.483 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-04-25T11:53:46.483 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-04-25T11:53:46.483 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.483 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.483 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.483 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-04-25T11:53:46.483 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.484 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.484 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.484 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.484 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.484 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.484 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.484 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-04-25T11:53:46.484 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-04-25T11:53:46.485 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.485 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.485 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.485 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.485 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.485 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.485 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.485 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.486 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.486 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.486 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.486 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-04-25T11:53:46.486 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-04-25T11:53:46.486 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.486 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.487 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.487 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.487 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.487 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.487 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.487 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.487 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.487 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.487 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.488 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-04-25T11:53:46.488 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-04-25T11:53:46.488 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:46.488 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.488 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.488 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.488 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.489 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.489 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.489 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.489 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.489 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.489 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.489 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-04-25T11:53:46.489 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-04-25T11:53:46.490 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.490 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.490 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.490 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.490 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.490 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.494 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.494 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.494 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.495 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.495 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.495 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-04-25T11:53:46.495 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-04-25T11:53:46.495 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.495 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.495 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.496 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.496 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.496 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.496 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.496 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.496 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.496 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.496 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.497 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-04-25T11:53:46.497 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-04-25T11:53:46.497 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.497 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.497 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.497 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.497 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.497 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-04-25T11:53:46.498 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-04-25T11:53:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-25T11:53:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-04-25T11:53:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.499 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.500 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.500 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.500 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.500 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-04-25T11:53:46.500 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-04-25T11:53:46.500 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.500 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.500 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.501 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.501 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.501 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.501 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.501 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.501 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.501 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.501 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.501 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.502 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-04-25T11:53:46.502 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-04-25T11:53:46.502 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.502 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.502 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.502 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.502 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.502 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.503 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.503 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.503 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.503 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.503 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.503 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.503 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-04-25T11:53:46.503 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-04-25T11:53:46.503 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.504 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.504 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.504 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.504 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.504 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.504 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.504 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.504 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.505 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.505 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.505 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.505 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.505 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.505 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-04-25T11:53:46.505 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.506 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.506 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.506 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-04-25T11:53:46.506 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-04-25T11:53:46.506 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.506 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.506 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.506 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.507 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.507 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.507 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.507 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T11:53:46.507 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.507 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.507 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.507 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-04-25T11:53:46.508 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-04-25T11:53:46.508 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.508 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.508 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.508 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.508 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.508 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.508 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.509 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-04-25T11:53:46.509 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.509 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.509 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.509 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.509 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.509 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.509 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.510 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.510 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.510 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.510 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.510 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.510 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.510 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.510 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.510 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.511 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.511 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.511 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.511 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.511 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.511 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.511 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.512 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.512 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.512 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.512 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.512 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.512 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.512 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.512 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.512 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.513 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.513 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.513 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.513 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.513 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.513 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.513 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.513 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.514 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.514 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.514 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.514 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.514 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.514 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.514 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.514 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.515 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.515 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.515 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.515 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.515 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.515 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.515 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.515 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.516 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.516 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.516 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.516 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.516 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.516 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.516 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.516 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.516 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.517 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.517 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.517 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.517 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.517 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.517 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.517 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.518 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-04-25T11:53:46.518 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.518 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.518 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.518 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.518 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.518 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.518 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.518 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.519 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.519 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.519 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.519 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.519 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.519 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.519 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.519 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.519 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.520 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.520 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.520 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.520 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.520 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.520 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.520 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.520 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.521 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.521 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.521 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.521 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.521 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.521 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.521 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.521 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.521 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.522 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.522 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.522 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.522 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.522 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.522 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.522 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.523 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.523 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.523 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.523 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.523 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.523 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.523 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.523 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.523 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.524 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.524 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.524 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.524 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.524 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.524 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.524 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.525 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.525 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.525 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.525 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.525 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.525 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.525 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.525 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.525 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.526 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.526 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.526 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.526 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.526 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.526 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.526 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.526 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-04-25T11:53:46.527 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.527 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.527 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.527 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.527 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.527 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.527 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.528 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.528 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.528 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.528 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.528 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.528 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.528 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.528 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.529 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.529 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.529 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.529 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.529 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.529 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.529 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.529 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.529 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.530 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.530 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.530 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.530 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.530 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.530 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.530 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.530 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.531 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.531 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.531 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.531 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.531 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.531 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.531 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.531 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.532 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.532 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.532 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.532 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.532 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.532 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.532 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.532 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.533 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.533 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.533 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.533 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.533 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.533 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.533 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.533 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.533 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.534 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.534 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.534 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.534 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.534 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.534 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.534 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.535 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.535 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.535 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.535 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.535 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.535 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.535 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.535 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.536 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-04-25T11:53:46.536 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.536 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.536 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.536 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-04-25T11:53:46.536 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-04-25T11:53:46.536 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.537 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.537 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.537 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T11:53:46.537 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.537 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.537 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.537 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.537 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.537 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.538 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.538 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-25T11:53:46.538 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-25T11:53:46.538 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.538 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.538 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.538 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-04-25T11:53:46.539 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.539 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.539 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.539 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.539 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.539 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.539 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.539 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.539 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.540 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.540 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.540 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.540 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.540 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.540 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.540 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.540 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.540 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.541 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.541 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.541 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.541 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.541 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.541 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.541 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.541 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.542 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.542 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.542 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.542 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.542 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.542 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.542 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.542 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.543 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.543 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.543 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.543 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.543 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.543 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.543 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.544 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.544 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.544 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.544 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.544 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.544 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.544 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.544 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.544 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.545 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.545 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.545 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.545 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.545 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.545 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.545 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.545 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.545 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.546 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.546 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.546 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.546 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.546 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.546 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.546 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.547 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.547 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.547 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.547 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.547 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.547 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.547 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.547 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.547 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.548 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.548 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-04-25T11:53:46.548 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.548 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.548 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.548 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-04-25T11:53:46.548 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-04-25T11:53:46.549 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.549 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.549 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.549 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.549 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.549 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.549 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.549 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T11:53:46.549 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.550 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.550 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.550 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-04-25T11:53:46.550 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-04-25T11:53:46.550 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.550 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.550 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.551 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.551 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.551 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.551 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.551 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T11:53:46.551 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.551 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.551 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.552 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-04-25T11:53:46.552 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-04-25T11:53:46.552 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.552 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.552 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.552 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.552 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.553 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.553 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.553 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T11:53:46.553 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.553 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.553 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.553 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-04-25T11:53:46.553 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-04-25T11:53:46.554 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.554 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.554 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.554 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.554 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.554 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.554 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.554 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T11:53:46.555 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.555 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.555 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.555 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.555 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-04-25T11:53:46.555 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-04-25T11:53:46.555 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.555 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.555 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.556 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.556 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.556 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.556 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.556 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T11:53:46.556 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.556 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.556 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.557 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.557 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-04-25T11:53:46.557 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-04-25T11:53:46.557 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.557 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.557 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.557 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.557 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.557 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.558 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.558 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.558 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.558 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.558 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.558 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-04-25T11:53:46.558 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-04-25T11:53:46.558 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.559 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.559 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.559 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.559 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.559 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.559 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.559 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.559 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.560 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.560 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.560 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-04-25T11:53:46.560 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-04-25T11:53:46.560 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.560 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.560 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.561 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T11:53:46.561 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.561 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.561 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.561 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.561 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.561 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.561 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.562 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-25T11:53:46.562 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-25T11:53:46.562 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.562 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.562 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.562 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.562 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.562 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.562 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.563 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.563 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.563 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.563 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.563 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-25T11:53:46.563 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-25T11:53:46.563 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.564 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.564 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.564 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-04-25T11:53:46.564 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-04-25T11:53:46.564 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.564 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.564 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.564 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.564 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.565 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.565 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-04-25T11:53:46.565 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-04-25T11:53:46.565 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.565 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.565 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.565 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-04-25T11:53:46.565 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.566 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.566 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.566 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.566 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.566 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.566 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.566 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-04-25T11:53:46.567 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-04-25T11:53:46.567 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.567 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.567 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.567 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.567 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.567 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.567 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.567 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.568 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.568 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.568 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.568 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.568 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-04-25T11:53:46.568 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-04-25T11:53:46.568 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.569 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.569 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.569 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.569 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.569 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.569 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.569 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.569 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.569 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.570 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.570 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.570 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.570 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.570 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.570 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.570 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.570 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.571 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.571 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.571 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.571 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.571 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.571 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.571 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.571 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.572 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.572 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.572 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.572 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.572 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.572 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.572 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.573 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.573 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.573 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.573 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.573 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.573 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.573 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.573 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.573 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.574 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.574 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.574 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.574 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.574 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.574 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.574 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.575 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.575 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.575 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.575 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.575 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.575 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.575 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.575 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.575 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.576 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.576 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.576 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.576 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.576 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.576 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.576 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.576 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.577 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.577 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.577 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.577 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.577 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.577 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.577 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.577 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.577 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.578 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.578 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.578 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-04-25T11:53:46.578 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-04-25T11:53:46.578 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.578 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.578 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.579 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.579 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.579 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.579 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.579 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.579 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.579 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.579 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.580 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-04-25T11:53:46.580 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-04-25T11:53:46.580 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.580 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.580 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.580 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.580 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.580 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.581 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.581 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.581 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.581 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.581 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.581 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-04-25T11:53:46.581 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-04-25T11:53:46.581 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.582 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.582 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.582 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T11:53:46.582 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.582 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.582 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.582 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.582 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.583 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.583 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.583 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.583 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.583 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.583 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-04-25T11:53:46.583 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.583 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.584 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.584 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.584 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.584 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.584 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.584 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.584 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.584 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.584 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.585 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.585 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.585 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.585 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.585 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.585 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.585 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.585 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.586 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.586 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.586 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.586 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.586 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.586 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.586 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.586 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.586 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.587 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.587 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.587 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.587 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.587 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.587 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.587 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.587 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.587 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.588 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.588 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.588 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.588 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.588 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.588 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.588 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.589 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.589 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.589 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.589 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.589 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.589 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.589 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.589 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.589 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.590 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.590 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.590 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.590 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.590 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.590 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.590 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.591 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.591 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.591 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.591 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.591 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.591 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.591 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.591 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.591 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.592 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.592 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.592 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.592 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.592 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.592 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-04-25T11:53:46.592 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.592 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.593 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.593 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.593 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.593 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.593 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.593 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.593 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.593 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.594 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.594 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.594 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.594 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.594 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.594 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.594 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.594 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.595 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.595 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.595 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.595 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.595 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.595 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.595 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.595 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.596 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.596 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.596 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.596 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.596 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.596 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.596 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.596 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.597 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.597 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.597 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.597 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.597 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.597 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.597 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.597 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.598 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.598 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.598 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.598 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.598 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.598 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.598 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.598 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.599 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.599 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.599 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.599 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.599 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.599 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.599 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.599 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.600 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.600 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.600 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.600 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.600 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.600 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.600 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.600 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.601 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.601 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.601 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.601 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-04-25T11:53:46.601 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-04-25T11:53:46.601 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.601 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.602 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.602 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-04-25T11:53:46.602 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.602 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.602 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.602 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T11:53:46.602 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.602 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.602 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.603 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-04-25T11:53:46.603 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-04-25T11:53:46.603 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.603 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.603 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.603 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.603 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.604 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.604 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.604 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-04-25T11:53:46.604 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.604 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.604 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.604 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.604 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-04-25T11:53:46.604 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-04-25T11:53:46.605 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.605 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.605 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.605 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-25T11:53:46.605 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.605 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.605 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.605 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T11:53:46.605 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.606 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.606 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.606 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_no_wait": { 2024-04-25T11:53:46.606 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T11:53:46.606 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.606 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.606 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.607 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.607 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.607 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.607 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.607 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T11:53:46.607 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.607 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.607 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.607 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.608 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.608 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.608 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-04-25T11:53:46.608 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.608 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.608 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.608 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-04-25T11:53:46.609 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-04-25T11:53:46.609 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-25T11:53:46.609 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.609 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.609 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-04-25T11:53:46.609 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.609 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.609 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.609 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.610 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.610 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.610 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.610 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-04-25T11:53:46.610 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-04-25T11:53:46.610 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.610 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.611 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.611 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.611 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.611 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.611 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.611 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.611 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.611 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.611 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.612 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-25T11:53:46.612 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-25T11:53:46.612 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.612 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.612 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.612 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-25T11:53:46.612 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.612 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.613 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.613 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.613 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.613 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.613 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.613 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.613 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.613 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.614 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.614 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.614 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.614 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.614 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.614 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.614 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.615 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.615 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.615 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.615 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.615 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.615 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.615 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.615 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.616 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.616 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.616 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.616 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.616 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.616 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.616 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.616 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.616 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.617 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.617 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.617 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.617 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.617 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.617 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.617 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.617 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.618 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.618 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.618 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.618 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.618 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.618 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.618 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.619 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.619 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.619 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.619 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.619 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.619 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.619 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.619 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.619 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.620 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.620 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.620 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.620 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.620 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.620 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.620 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.621 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.621 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.621 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.621 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.621 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.621 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.621 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.621 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.622 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-04-25T11:53:46.622 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-04-25T11:53:46.622 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.622 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.622 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.622 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.622 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.622 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.623 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.623 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.623 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.623 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.623 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.623 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-04-25T11:53:46.623 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-04-25T11:53:46.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.624 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.624 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.624 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-04-25T11:53:46.624 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.624 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.624 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.624 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.624 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.625 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.625 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.625 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.625 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-04-25T11:53:46.625 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-04-25T11:53:46.625 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.625 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.625 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.625 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T11:53:46.626 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.626 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.626 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.626 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.626 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.626 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.626 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.626 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.627 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.627 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.627 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T11:53:46.627 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.627 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.627 INFO:teuthology.orchestra.run.smithi083.stdout: "modules": [ 2024-04-25T11:53:46.627 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-04-25T11:53:46.627 INFO:teuthology.orchestra.run.smithi083.stdout: "dashboard", 2024-04-25T11:53:46.627 INFO:teuthology.orchestra.run.smithi083.stdout: "iostat", 2024-04-25T11:53:46.628 INFO:teuthology.orchestra.run.smithi083.stdout: "nfs", 2024-04-25T11:53:46.628 INFO:teuthology.orchestra.run.smithi083.stdout: "restful", 2024-04-25T11:53:46.628 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_schedule" 2024-04-25T11:53:46.628 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.628 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-04-25T11:53:46.628 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.628 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-04-25T11:53:46.628 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.628 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.629 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.629 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-25T11:53:46.629 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-25T11:53:46.629 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.629 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.629 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.629 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-25T11:53:46.629 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.630 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.630 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.630 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T11:53:46.630 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.630 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.630 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.630 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.630 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.631 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.631 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.631 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.631 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.631 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.631 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.631 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.631 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.632 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.632 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.632 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.632 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.632 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.632 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.632 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.632 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.632 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.633 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.633 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.633 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.633 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.633 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.633 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.633 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.633 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.634 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.634 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.634 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.634 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.634 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.634 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.634 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.635 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.635 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.635 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.635 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.635 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.635 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.635 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.635 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.636 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.636 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.636 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.636 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.636 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.636 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.636 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.636 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.637 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.637 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.637 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.637 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.637 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.637 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.637 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.638 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.638 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.638 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.638 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.638 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.638 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.638 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.638 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.639 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-04-25T11:53:46.639 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-04-25T11:53:46.639 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.639 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.639 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.639 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.639 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.639 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.639 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.640 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2024-04-25T11:53:46.640 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.640 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.640 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.640 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-04-25T11:53:46.640 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-04-25T11:53:46.640 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.641 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.641 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.641 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-04-25T11:53:46.641 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.641 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.641 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.641 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-04-25T11:53:46.641 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.641 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.642 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.642 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-04-25T11:53:46.642 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-04-25T11:53:46.642 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.642 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.642 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.642 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.643 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.643 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.643 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.643 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-04-25T11:53:46.643 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.643 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.643 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.643 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-04-25T11:53:46.644 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-04-25T11:53:46.644 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.644 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.644 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.644 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.644 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.644 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.644 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.645 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-04-25T11:53:46.645 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.645 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.645 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.645 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.645 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-04-25T11:53:46.645 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-04-25T11:53:46.645 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.646 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.646 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.646 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-04-25T11:53:46.646 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.646 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.646 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.646 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-04-25T11:53:46.646 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.647 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.647 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.647 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-04-25T11:53:46.647 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-04-25T11:53:46.647 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.647 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.647 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.647 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.648 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.648 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.648 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.648 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-04-25T11:53:46.648 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.648 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.648 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.649 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-04-25T11:53:46.649 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-04-25T11:53:46.649 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.649 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.649 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.649 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.649 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.649 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.649 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.650 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T11:53:46.650 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.650 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.650 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.650 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-04-25T11:53:46.650 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-04-25T11:53:46.650 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.651 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.651 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.651 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.651 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.651 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.651 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.651 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-04-25T11:53:46.651 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.652 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.652 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.652 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.652 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.652 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.652 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-04-25T11:53:46.652 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.652 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.653 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.653 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-04-25T11:53:46.653 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-04-25T11:53:46.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.653 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.653 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.653 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.653 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.654 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.654 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T11:53:46.654 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.654 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.654 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.654 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-04-25T11:53:46.654 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-04-25T11:53:46.655 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.655 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.655 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.655 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-04-25T11:53:46.655 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.655 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.655 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.655 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T11:53:46.655 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:53:46.656 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.656 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.656 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-04-25T11:53:46.656 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-04-25T11:53:46.656 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-25T11:53:46.656 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.656 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.657 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-25T11:53:46.657 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-25T11:53:46.657 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-25T11:53:46.657 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.657 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T11:53:46.657 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:53:46.657 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.657 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.658 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-04-25T11:53:46.658 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-04-25T11:53:46.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-25T11:53:46.658 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.658 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-04-25T11:53:46.658 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-25T11:53:46.658 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-04-25T11:53:46.659 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.659 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T11:53:46.659 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.659 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.659 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-04-25T11:53:46.659 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-04-25T11:53:46.659 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.660 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.660 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.660 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T11:53:46.660 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.660 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.660 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.660 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T11:53:46.660 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T11:53:46.660 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.661 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.661 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-04-25T11:53:46.661 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-04-25T11:53:46.661 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:46.661 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.661 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.661 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-04-25T11:53:46.662 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-04-25T11:53:46.662 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-04-25T11:53:46.662 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.662 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-04-25T11:53:46.662 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T11:53:46.662 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.662 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.662 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-04-25T11:53:46.663 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-04-25T11:53:46.663 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.663 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.663 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.663 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-04-25T11:53:46.663 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.663 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.663 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.664 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T11:53:46.664 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T11:53:46.664 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.664 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.664 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-04-25T11:53:46.664 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-04-25T11:53:46.664 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-25T11:53:46.664 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.665 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.665 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-25T11:53:46.665 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-25T11:53:46.665 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-25T11:53:46.665 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.665 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T11:53:46.665 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T11:53:46.665 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.665 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.666 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.666 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.666 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.666 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.666 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.666 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.666 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.667 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.667 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.667 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.667 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.667 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.667 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.667 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.667 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.668 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.668 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.668 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.668 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.668 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.668 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.668 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.669 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.669 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.669 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.669 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.669 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.669 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.669 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.669 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.670 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.670 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.670 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.670 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.670 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.670 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.670 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.671 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.671 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.671 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.671 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.671 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.671 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.671 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.671 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.672 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.672 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.672 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.672 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.672 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.672 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.672 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.672 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.673 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.673 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.673 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.673 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.673 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.673 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.673 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.673 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.674 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.674 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.674 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.674 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.674 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.674 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-04-25T11:53:46.674 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-04-25T11:53:46.675 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:46.675 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.675 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.675 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-25T11:53:46.675 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.675 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.675 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.675 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T11:53:46.675 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.676 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.676 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.676 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-25T11:53:46.676 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-25T11:53:46.676 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.676 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.676 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.677 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-04-25T11:53:46.677 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.677 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.677 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.677 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-04-25T11:53:46.677 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-04-25T11:53:46.677 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-04-25T11:53:46.677 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-04-25T11:53:46.677 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-04-25T11:53:46.678 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.678 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-04-25T11:53:46.678 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.678 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.678 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.678 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-04-25T11:53:46.678 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-04-25T11:53:46.679 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.679 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.679 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.679 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.679 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.679 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.679 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.679 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T11:53:46.680 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.680 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.680 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.680 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-25T11:53:46.680 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-25T11:53:46.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.680 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.680 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.681 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-25T11:53:46.681 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.681 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.681 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.681 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T11:53:46.681 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.681 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.681 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.682 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-04-25T11:53:46.682 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-04-25T11:53:46.682 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.682 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.682 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.682 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-25T11:53:46.682 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-25T11:53:46.682 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.683 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.683 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T11:53:46.683 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T11:53:46.683 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.683 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.683 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-04-25T11:53:46.683 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-04-25T11:53:46.683 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-25T11:53:46.684 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.684 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.684 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:46.684 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.684 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.684 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.684 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T11:53:46.684 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.684 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.685 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.685 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.685 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.685 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-04-25T11:53:46.685 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.685 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.686 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.686 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-04-25T11:53:46.686 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-04-25T11:53:46.686 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:46.686 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.686 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.686 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-25T11:53:46.686 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.686 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.687 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.687 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T11:53:46.687 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.687 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.687 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.687 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.687 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-04-25T11:53:46.687 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-04-25T11:53:46.687 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.688 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.688 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.688 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-04-25T11:53:46.688 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.688 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.688 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.688 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T11:53:46.688 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.689 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.689 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.689 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-04-25T11:53:46.689 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-04-25T11:53:46.689 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.689 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.689 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.689 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-04-25T11:53:46.690 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.690 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.690 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.690 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T11:53:46.690 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.690 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.690 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.690 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-04-25T11:53:46.691 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-04-25T11:53:46.691 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.691 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.691 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.691 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.691 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.691 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.691 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.692 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T11:53:46.692 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T11:53:46.692 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.692 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.692 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-04-25T11:53:46.692 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-04-25T11:53:46.692 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.692 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.693 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.693 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-25T11:53:46.693 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.693 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.693 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.693 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T11:53:46.693 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.693 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.693 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.694 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-04-25T11:53:46.694 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T11:53:46.694 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:46.694 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.694 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.698 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-04-25T11:53:46.698 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.698 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.698 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.698 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T11:53:46.698 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.699 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.699 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.699 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-04-25T11:53:46.699 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-04-25T11:53:46.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.699 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.699 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.699 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.700 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.700 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.700 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.700 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-04-25T11:53:46.700 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-04-25T11:53:46.700 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-04-25T11:53:46.700 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.700 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T11:53:46.701 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.701 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.701 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.701 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-04-25T11:53:46.701 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-04-25T11:53:46.701 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.701 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.701 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.702 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.702 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.702 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.702 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.702 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T11:53:46.702 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.702 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.702 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.703 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-04-25T11:53:46.703 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-04-25T11:53:46.703 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.703 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.703 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.703 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.703 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.703 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.704 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.704 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T11:53:46.704 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.704 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.704 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.704 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-04-25T11:53:46.704 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-04-25T11:53:46.705 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.705 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.705 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.705 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.705 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.705 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.705 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.705 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T11:53:46.705 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.706 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.706 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.706 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-04-25T11:53:46.706 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-04-25T11:53:46.706 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.706 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.706 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.706 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T11:53:46.707 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.707 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.707 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.707 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-25T11:53:46.707 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.707 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.707 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.708 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-04-25T11:53:46.708 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-04-25T11:53:46.708 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.708 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.708 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.708 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T11:53:46.708 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.708 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.708 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.709 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-04-25T11:53:46.709 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.709 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.709 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.709 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-04-25T11:53:46.709 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-04-25T11:53:46.709 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.709 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.710 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.710 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T11:53:46.710 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.710 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.710 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.710 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-04-25T11:53:46.710 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.710 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.711 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.711 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-04-25T11:53:46.711 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-04-25T11:53:46.711 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.711 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.711 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.711 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T11:53:46.711 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.712 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.712 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.712 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-25T11:53:46.712 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.712 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.712 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.712 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-04-25T11:53:46.713 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-04-25T11:53:46.713 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.713 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.713 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.713 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T11:53:46.713 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.713 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.713 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.713 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-04-25T11:53:46.714 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.714 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.714 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.714 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-04-25T11:53:46.714 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-04-25T11:53:46.714 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.714 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.715 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.715 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T11:53:46.715 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.715 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.715 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.715 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-04-25T11:53:46.715 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.715 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.715 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.716 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-04-25T11:53:46.716 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-04-25T11:53:46.716 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.716 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.716 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.716 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T11:53:46.716 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.716 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.717 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.717 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-04-25T11:53:46.717 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.717 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.717 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.717 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-04-25T11:53:46.717 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-04-25T11:53:46.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.718 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.718 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.718 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T11:53:46.718 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.718 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.718 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.718 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-04-25T11:53:46.719 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.719 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.719 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.719 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-04-25T11:53:46.719 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-04-25T11:53:46.719 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.719 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.719 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.720 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T11:53:46.720 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.720 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.720 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.720 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-04-25T11:53:46.720 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.720 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.720 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.721 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.721 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-04-25T11:53:46.721 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-04-25T11:53:46.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.721 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.721 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.721 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T11:53:46.721 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.722 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.722 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.722 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-04-25T11:53:46.722 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.722 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.722 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.722 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-04-25T11:53:46.723 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-04-25T11:53:46.723 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.723 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.723 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.723 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T11:53:46.723 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.723 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.723 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.724 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-25T11:53:46.724 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.724 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.724 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.724 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-04-25T11:53:46.724 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-04-25T11:53:46.724 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.724 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.725 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.725 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T11:53:46.725 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.725 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.725 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.725 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-04-25T11:53:46.725 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.725 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.726 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.726 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-04-25T11:53:46.726 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-04-25T11:53:46.726 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.726 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.726 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.726 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T11:53:46.726 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.727 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.727 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.727 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-25T11:53:46.727 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.727 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.727 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.727 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-04-25T11:53:46.728 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-04-25T11:53:46.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.728 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.728 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.728 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T11:53:46.728 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.728 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.728 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.728 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-04-25T11:53:46.729 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.729 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.729 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.729 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.729 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba": { 2024-04-25T11:53:46.729 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba", 2024-04-25T11:53:46.729 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.729 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.729 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.730 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T11:53:46.730 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.730 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.730 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.730 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB container image", 2024-04-25T11:53:46.730 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.730 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.730 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.731 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-04-25T11:53:46.731 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-04-25T11:53:46.731 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.731 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.731 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.731 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T11:53:46.731 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.731 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.732 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.732 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-04-25T11:53:46.732 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.732 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.732 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.732 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-04-25T11:53:46.732 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-04-25T11:53:46.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.733 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.733 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.733 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.733 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.733 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.733 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.733 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T11:53:46.733 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.734 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.734 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.734 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-04-25T11:53:46.734 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-04-25T11:53:46.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.734 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.734 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-25T11:53:46.735 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.735 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.735 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.735 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T11:53:46.735 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.735 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.735 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.736 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-04-25T11:53:46.736 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T11:53:46.736 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.736 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.736 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.736 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-04-25T11:53:46.736 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.736 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.737 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.737 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T11:53:46.737 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.737 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.737 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.737 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-04-25T11:53:46.737 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-04-25T11:53:46.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.738 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.738 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.738 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-04-25T11:53:46.738 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.738 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.738 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.738 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T11:53:46.739 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.739 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.739 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.739 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-04-25T11:53:46.739 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-04-25T11:53:46.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.739 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.739 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.740 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-04-25T11:53:46.740 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.740 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.740 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.740 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-04-25T11:53:46.740 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.740 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.740 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.741 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.741 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-04-25T11:53:46.741 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-04-25T11:53:46.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.741 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.741 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.741 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.741 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.741 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.742 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.742 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T11:53:46.742 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.742 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.742 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.742 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-04-25T11:53:46.742 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-04-25T11:53:46.743 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.743 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.743 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.743 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-25T11:53:46.743 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.743 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.743 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.743 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-04-25T11:53:46.744 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.744 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.744 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.744 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.744 INFO:teuthology.orchestra.run.smithi083.stdout: "grafana_dashboards_path": { 2024-04-25T11:53:46.744 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "grafana_dashboards_path", 2024-04-25T11:53:46.744 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.744 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.744 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.745 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T11:53:46.745 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.745 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.745 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.745 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T11:53:46.745 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.745 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.745 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.746 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-04-25T11:53:46.746 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-04-25T11:53:46.746 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.746 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.746 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.746 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-25T11:53:46.746 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.747 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.747 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.747 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-04-25T11:53:46.747 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.747 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.747 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.747 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.747 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-04-25T11:53:46.747 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-04-25T11:53:46.748 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.748 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.748 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.748 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.748 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.748 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.748 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.748 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T11:53:46.748 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.749 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.749 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.749 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-04-25T11:53:46.749 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-04-25T11:53:46.749 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.749 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.749 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.750 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.750 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.750 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.750 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.750 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T11:53:46.750 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.750 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.750 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.750 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.751 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.751 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.751 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.751 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.751 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.751 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.751 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.751 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.752 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.752 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.752 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.752 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.752 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.752 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.752 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.752 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.753 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.753 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.753 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.753 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-04-25T11:53:46.753 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-04-25T11:53:46.753 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.753 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.753 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.754 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.754 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.754 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.754 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.754 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T11:53:46.754 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.754 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.754 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.755 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.755 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.755 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.755 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.755 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.755 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.755 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.755 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.755 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.756 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.756 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T11:53:46.756 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.756 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.756 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.756 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.756 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.757 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.757 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.757 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.757 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.757 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.757 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.757 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.757 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.758 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.758 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.758 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.758 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.758 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.758 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.758 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.758 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.758 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.759 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.759 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.759 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.759 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.759 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.759 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.759 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.760 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.760 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.760 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.760 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.760 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.760 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.760 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T11:53:46.760 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T11:53:46.761 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.761 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.761 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.761 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.761 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.761 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.761 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T11:53:46.761 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.762 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.762 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.762 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T11:53:46.762 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T11:53:46.762 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.762 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.762 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.763 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-04-25T11:53:46.763 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.763 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.763 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.763 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T11:53:46.763 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.763 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.763 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.763 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.764 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-04-25T11:53:46.764 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-04-25T11:53:46.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.764 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.764 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.764 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:46.764 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.764 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.764 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.765 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-04-25T11:53:46.765 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.765 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.765 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.765 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.765 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-04-25T11:53:46.765 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-04-25T11:53:46.765 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.766 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.766 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.766 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:46.766 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.766 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.766 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.766 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T11:53:46.766 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.767 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.767 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.767 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.767 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-04-25T11:53:46.767 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-04-25T11:53:46.767 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.767 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.767 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.767 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.768 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.768 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.768 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.768 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-04-25T11:53:46.768 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.768 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.768 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.769 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-25T11:53:46.769 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-25T11:53:46.769 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.769 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.769 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.769 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-04-25T11:53:46.769 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.769 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.769 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.770 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-04-25T11:53:46.770 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-04-25T11:53:46.770 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.770 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T11:53:46.770 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.770 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.770 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.771 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-04-25T11:53:46.771 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-04-25T11:53:46.771 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.771 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.771 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.771 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-04-25T11:53:46.771 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.771 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.772 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.772 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T11:53:46.772 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.772 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.772 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.772 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-04-25T11:53:46.772 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-04-25T11:53:46.772 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.773 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.773 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.773 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T11:53:46.773 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.773 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.773 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.773 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T11:53:46.773 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.773 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.774 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.774 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-04-25T11:53:46.774 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-04-25T11:53:46.774 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.774 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.774 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.774 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.775 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.775 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.775 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.775 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T11:53:46.775 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.775 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.775 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.775 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-04-25T11:53:46.776 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-04-25T11:53:46.776 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.776 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.776 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.776 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.776 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.776 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.776 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.777 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T11:53:46.777 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.777 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.777 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.777 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-04-25T11:53:46.777 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-04-25T11:53:46.777 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.777 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.778 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.778 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.778 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.778 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.778 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.778 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T11:53:46.778 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.778 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.779 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.779 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-04-25T11:53:46.779 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-04-25T11:53:46.779 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.779 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.779 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.779 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.779 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.780 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.780 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.780 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T11:53:46.780 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.780 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.780 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.780 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-04-25T11:53:46.780 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-04-25T11:53:46.781 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.781 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.781 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.781 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.781 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.781 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.781 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.781 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T11:53:46.782 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.782 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.782 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.782 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-04-25T11:53:46.782 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-04-25T11:53:46.782 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.782 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.782 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.783 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-04-25T11:53:46.783 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.783 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.783 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.783 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-04-25T11:53:46.783 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.783 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.783 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.784 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-04-25T11:53:46.784 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-04-25T11:53:46.784 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.784 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.784 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.784 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.784 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.784 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.785 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.785 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T11:53:46.785 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.785 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.785 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.785 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-04-25T11:53:46.785 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-04-25T11:53:46.785 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.786 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.786 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.786 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.786 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.786 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.786 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.786 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T11:53:46.786 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.787 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.787 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.787 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-04-25T11:53:46.787 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-04-25T11:53:46.787 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.787 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.787 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.787 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.788 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.788 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.788 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.788 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T11:53:46.788 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.788 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.788 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.788 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.788 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-04-25T11:53:46.789 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-04-25T11:53:46.789 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.789 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.789 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.789 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.789 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.789 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.789 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.790 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T11:53:46.790 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.790 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.790 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.790 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-04-25T11:53:46.790 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-04-25T11:53:46.790 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.790 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.791 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.791 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.791 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.791 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.791 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.791 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T11:53:46.791 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.791 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.792 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.792 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-04-25T11:53:46.792 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-04-25T11:53:46.792 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.792 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.792 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.792 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.792 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.793 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.793 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.793 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T11:53:46.793 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.793 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.793 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.793 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.793 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.794 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.794 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-04-25T11:53:46.794 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.794 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.794 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.794 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.794 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.794 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.795 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.795 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.795 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.795 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.795 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.795 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.795 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.795 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.796 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.796 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.796 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.796 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.796 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.796 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.796 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.796 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.796 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.797 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.797 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.797 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.797 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.797 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.797 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.797 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.798 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.798 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.798 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.798 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.798 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.798 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.798 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.798 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.799 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.799 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.799 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.799 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.799 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.799 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.799 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.799 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.800 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.800 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.800 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.800 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.800 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.800 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.800 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.800 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.800 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.801 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.801 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.801 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.801 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.801 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.801 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.801 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.801 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.802 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.802 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.802 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.802 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.802 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.802 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.802 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-04-25T11:53:46.803 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-04-25T11:53:46.803 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.803 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.803 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.803 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-04-25T11:53:46.803 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.803 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.803 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.803 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T11:53:46.804 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.804 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.804 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.804 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-04-25T11:53:46.804 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-04-25T11:53:46.804 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.804 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.804 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.805 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-04-25T11:53:46.805 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.805 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.805 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.805 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T11:53:46.805 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.805 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.805 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.806 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.806 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.806 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-04-25T11:53:46.806 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.806 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.806 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.806 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T11:53:46.807 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T11:53:46.807 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.807 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.807 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.807 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:46.807 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.807 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.807 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.808 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.808 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.808 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.808 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.808 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T11:53:46.808 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T11:53:46.808 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.808 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.809 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.809 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.809 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.809 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.809 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.809 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.809 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.809 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.809 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.810 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T11:53:46.810 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T11:53:46.810 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.810 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.810 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.810 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.810 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.811 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.811 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.811 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.811 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.811 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.811 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.811 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-04-25T11:53:46.811 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T11:53:46.812 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.812 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.812 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.812 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.812 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.812 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.812 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.812 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.812 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.813 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.813 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.813 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T11:53:46.813 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T11:53:46.813 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.813 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.813 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.813 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.814 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.814 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.814 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.814 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.814 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.814 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.814 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.815 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T11:53:46.815 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T11:53:46.815 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.815 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.815 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.815 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.815 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.815 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.815 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.816 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.816 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.816 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.816 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.816 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T11:53:46.816 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T11:53:46.816 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.817 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.817 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.817 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.817 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.817 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.817 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.817 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.817 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.817 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.818 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.818 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T11:53:46.818 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T11:53:46.818 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.818 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.818 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.818 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.818 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.819 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.819 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.819 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.819 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.819 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.819 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.819 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T11:53:46.820 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T11:53:46.820 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.820 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.820 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.820 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.820 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.820 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.820 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.820 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.821 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.821 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.821 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.821 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T11:53:46.821 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T11:53:46.821 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.821 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.822 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.822 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.822 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.822 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.822 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.822 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.822 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.822 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.823 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.823 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T11:53:46.823 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T11:53:46.823 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.823 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.823 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.823 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.823 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.824 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.824 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.824 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.824 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.824 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.824 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.824 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T11:53:46.824 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T11:53:46.825 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.825 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.825 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.825 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.825 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.825 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.825 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.825 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.826 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.826 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.826 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.826 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T11:53:46.826 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T11:53:46.826 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.826 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.826 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.827 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.827 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.827 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.827 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.827 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.827 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.827 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.827 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.828 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T11:53:46.828 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T11:53:46.828 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.828 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.828 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.828 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.828 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.828 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.829 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.829 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.829 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.829 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.829 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.829 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.829 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T11:53:46.829 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T11:53:46.830 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.830 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.830 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.830 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-25T11:53:46.830 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.830 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.830 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.830 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.830 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.831 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.831 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.831 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T11:53:46.831 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T11:53:46.831 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.831 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.831 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.832 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.832 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.832 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.832 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.832 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.832 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.832 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.832 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.833 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-04-25T11:53:46.833 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-04-25T11:53:46.833 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.833 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.833 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.833 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.833 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.833 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.834 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.834 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.834 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.834 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.834 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.834 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T11:53:46.834 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T11:53:46.834 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.835 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.835 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.835 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-25T11:53:46.835 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.835 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.835 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.835 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.835 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.835 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.836 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.836 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T11:53:46.836 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T11:53:46.836 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.836 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.836 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.836 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.837 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.837 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.837 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.837 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.837 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.837 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.837 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.837 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T11:53:46.838 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T11:53:46.838 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.838 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.838 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.838 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.838 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.838 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.838 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.839 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.839 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.839 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.839 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.839 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T11:53:46.839 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T11:53:46.839 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.839 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.840 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.840 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.840 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.840 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.840 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.840 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.840 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.840 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.840 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.841 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T11:53:46.841 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T11:53:46.841 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.841 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.841 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.841 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.841 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.842 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.842 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.842 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.842 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.842 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.842 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.842 INFO:teuthology.orchestra.run.smithi083.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T11:53:46.843 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T11:53:46.843 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.843 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.843 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.843 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "[]", 2024-04-25T11:53:46.843 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.843 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.843 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.843 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.844 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.844 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.844 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.844 INFO:teuthology.orchestra.run.smithi083.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T11:53:46.844 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T11:53:46.844 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.844 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.844 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.845 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "{}", 2024-04-25T11:53:46.845 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.845 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.845 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.845 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.845 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.845 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.845 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.846 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T11:53:46.846 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T11:53:46.846 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.846 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.846 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.846 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.846 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.847 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.847 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.847 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.847 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.847 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.847 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.847 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T11:53:46.847 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T11:53:46.848 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.848 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.848 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.848 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.848 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.848 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.848 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.848 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.849 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.849 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.849 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.849 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T11:53:46.849 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T11:53:46.849 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.849 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.850 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.850 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.850 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.850 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.850 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.850 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.850 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.850 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.850 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.851 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T11:53:46.851 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T11:53:46.851 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.851 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.851 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.851 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.851 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T11:53:46.852 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T11:53:46.853 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.853 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.853 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.853 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.853 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.853 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.853 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.853 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.853 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.854 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.854 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.854 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T11:53:46.854 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T11:53:46.854 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.854 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.854 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.855 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.855 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.855 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.855 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.855 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.855 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.855 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.855 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.856 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T11:53:46.856 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T11:53:46.856 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.856 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.856 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.856 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.856 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.856 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.856 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.857 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.857 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.857 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.857 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.857 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T11:53:46.857 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T11:53:46.857 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.858 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.858 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.858 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.858 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.858 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.858 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.858 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.858 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.859 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.859 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.859 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T11:53:46.859 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T11:53:46.859 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.859 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.859 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.859 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.860 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.860 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.860 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.860 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.860 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.860 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.860 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.860 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T11:53:46.861 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T11:53:46.861 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.861 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.861 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.861 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.861 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.861 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.861 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.862 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.862 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.862 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.862 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.862 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T11:53:46.862 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T11:53:46.862 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.862 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.863 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.863 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T11:53:46.863 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.863 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.863 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.863 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.863 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.863 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.864 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.864 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T11:53:46.864 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T11:53:46.864 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.864 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.864 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.864 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:46.864 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.865 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.865 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.865 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.865 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.865 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.865 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.865 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T11:53:46.866 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T11:53:46.866 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.866 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.866 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.866 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-04-25T11:53:46.866 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.866 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.866 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.866 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.867 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.867 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.867 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.867 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T11:53:46.867 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T11:53:46.867 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.867 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.867 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.868 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-04-25T11:53:46.868 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.868 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.868 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.868 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.868 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.868 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.868 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.869 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T11:53:46.869 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T11:53:46.869 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.869 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.869 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.869 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.869 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.870 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.870 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.870 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.870 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.870 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.870 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.870 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T11:53:46.870 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T11:53:46.871 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.871 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.871 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.871 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-25T11:53:46.871 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.871 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.871 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.871 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.872 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.872 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.872 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.872 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T11:53:46.872 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T11:53:46.872 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.872 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.872 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.873 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.873 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.873 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.873 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.873 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.873 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.873 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.873 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.874 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T11:53:46.874 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T11:53:46.874 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.874 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.874 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.874 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.874 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.875 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.875 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.875 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.875 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.875 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.875 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.875 INFO:teuthology.orchestra.run.smithi083.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T11:53:46.876 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T11:53:46.876 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.876 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.876 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.876 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.876 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.876 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.876 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.877 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.877 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.877 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.877 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.877 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T11:53:46.877 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T11:53:46.877 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.877 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.878 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.878 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-25T11:53:46.878 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.878 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.878 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.878 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.878 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.878 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.879 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.879 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T11:53:46.879 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T11:53:46.879 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.879 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.879 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.879 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:46.879 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.880 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.880 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.880 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.880 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.880 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.880 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.880 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T11:53:46.881 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T11:53:46.881 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.881 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.881 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.881 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-25T11:53:46.881 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.881 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.881 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.881 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.882 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.882 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.882 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.882 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-04-25T11:53:46.882 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-04-25T11:53:46.882 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.882 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.883 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.883 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.883 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.883 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.883 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.883 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.883 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.883 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.883 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.884 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-04-25T11:53:46.884 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-04-25T11:53:46.884 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.884 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.884 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.884 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.884 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.885 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.885 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.885 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.885 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.885 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.885 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.885 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-04-25T11:53:46.885 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-04-25T11:53:46.886 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.886 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.886 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.886 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.886 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.886 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.886 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.886 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-04-25T11:53:46.887 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.887 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.887 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.887 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-04-25T11:53:46.887 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-04-25T11:53:46.887 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.887 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.887 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.888 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-04-25T11:53:46.888 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.888 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.888 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.888 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.888 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.888 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.888 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.889 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-25T11:53:46.889 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-25T11:53:46.889 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.889 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.889 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.889 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.889 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.889 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.890 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.890 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.890 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.890 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.890 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.890 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.890 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.890 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.891 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.891 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.891 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.891 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.891 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.891 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.891 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.891 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.892 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.892 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.892 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.892 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.892 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.892 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.892 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.892 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.892 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.893 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.893 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.893 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.893 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.893 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.893 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.893 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.894 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.894 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.894 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.894 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.894 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.894 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.894 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.894 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.895 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.895 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.895 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.895 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.895 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.895 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.895 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.895 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.896 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.896 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.896 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.896 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.896 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.896 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.896 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.896 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.896 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.897 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.897 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.897 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.897 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.897 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.897 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.897 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.898 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.898 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.898 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.898 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.898 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.898 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.898 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.899 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-04-25T11:53:46.899 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-04-25T11:53:46.899 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.902 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.902 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.903 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.903 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.903 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.903 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.903 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-04-25T11:53:46.903 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.903 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.903 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.904 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-04-25T11:53:46.904 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T11:53:46.904 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.904 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.904 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.904 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.904 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.904 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.905 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.905 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.905 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.905 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.905 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.905 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.905 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-25T11:53:46.905 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-25T11:53:46.906 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.906 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.906 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.906 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-25T11:53:46.906 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.906 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.906 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.906 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.906 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.907 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.907 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.907 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.907 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-25T11:53:46.907 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-25T11:53:46.907 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.907 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.907 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.908 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-04-25T11:53:46.908 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.908 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.908 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.908 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.908 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.908 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.908 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.909 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-25T11:53:46.909 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-25T11:53:46.909 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.909 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.909 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.909 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.909 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.909 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.910 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.910 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.910 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.910 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.910 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.910 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-04-25T11:53:46.910 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-04-25T11:53:46.911 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.911 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.911 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.911 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-04-25T11:53:46.911 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.911 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.911 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.911 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.911 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.912 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.912 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.912 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.912 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-25T11:53:46.912 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-25T11:53:46.912 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.912 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.912 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.913 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-04-25T11:53:46.913 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.913 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.913 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.913 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.913 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-04-25T11:53:46.913 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.913 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-25T11:53:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-25T11:53:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.914 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-25T11:53:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-25T11:53:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-25T11:53:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.915 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-04-25T11:53:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-04-25T11:53:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.916 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.917 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.917 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.917 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.917 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.917 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.917 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.917 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.918 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.918 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-04-25T11:53:46.918 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.918 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.918 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.918 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-04-25T11:53:46.918 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-04-25T11:53:46.919 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.919 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.919 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.919 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.919 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.919 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.919 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.919 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-04-25T11:53:46.919 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.920 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.920 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.920 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.920 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.920 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.920 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.920 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.920 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.921 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.921 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.921 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.921 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.921 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.921 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.921 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.921 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.922 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.922 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.922 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.922 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.922 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.922 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.922 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.922 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.923 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.923 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.923 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.923 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.923 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.923 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.923 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.923 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.924 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.924 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.924 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.924 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.924 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.924 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.924 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.924 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.925 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.925 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.925 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.925 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.925 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.925 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.925 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.925 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.925 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.926 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.926 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.926 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.926 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.926 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.926 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.926 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.927 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.927 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.927 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.927 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.927 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.927 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.927 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.927 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.928 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.928 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.928 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.928 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.928 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-04-25T11:53:46.928 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-04-25T11:53:46.928 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.928 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.929 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.929 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-04-25T11:53:46.929 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.929 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.929 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.929 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T11:53:46.929 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.929 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.930 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.930 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.930 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-04-25T11:53:46.930 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-04-25T11:53:46.930 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.930 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.930 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.930 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-04-25T11:53:46.931 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.931 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.931 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.931 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T11:53:46.931 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.931 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.931 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.931 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-04-25T11:53:46.932 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-04-25T11:53:46.932 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.932 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.932 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.932 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-04-25T11:53:46.932 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.932 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.932 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.933 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-04-25T11:53:46.933 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.933 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.933 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.933 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-04-25T11:53:46.933 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-04-25T11:53:46.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.934 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.934 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.934 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-25T11:53:46.934 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.934 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.934 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.934 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T11:53:46.934 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.934 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.935 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.935 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-04-25T11:53:46.935 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-04-25T11:53:46.935 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.935 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.935 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.935 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:46.936 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.936 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.936 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.936 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T11:53:46.936 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.936 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.936 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.937 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-25T11:53:46.937 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-25T11:53:46.937 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.937 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.937 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.937 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-25T11:53:46.937 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.937 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.937 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.938 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T11:53:46.938 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.938 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.938 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.938 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-04-25T11:53:46.938 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-04-25T11:53:46.938 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.939 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.939 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.939 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-04-25T11:53:46.939 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.939 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.939 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.939 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T11:53:46.939 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.939 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.940 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.940 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.940 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.940 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.940 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-04-25T11:53:46.940 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.940 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.941 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.941 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.941 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.941 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.941 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.941 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.941 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.941 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.942 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.942 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.942 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.942 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.942 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.942 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.942 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.942 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.942 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.943 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.943 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.943 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.943 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.943 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.943 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.944 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.944 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.944 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.944 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.944 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.944 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.944 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.944 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.945 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.945 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.945 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.945 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.945 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.945 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.945 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.945 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.946 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.946 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.946 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.946 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.946 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.946 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.946 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.946 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.947 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.947 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.947 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.947 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.947 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.947 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.947 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.947 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.948 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.948 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.948 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.948 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.948 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.948 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.948 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.948 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.949 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.949 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.949 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.949 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-04-25T11:53:46.949 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-04-25T11:53:46.949 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.949 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.949 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.950 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-25T11:53:46.950 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.950 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.950 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.950 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.950 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.950 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.951 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.951 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-04-25T11:53:46.951 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-04-25T11:53:46.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.951 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.951 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.951 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-04-25T11:53:46.951 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.952 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.952 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.952 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.952 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.952 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.952 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.952 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-25T11:53:46.953 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-25T11:53:46.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.953 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.953 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.953 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-25T11:53:46.953 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.953 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.953 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.953 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.954 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.954 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.954 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.954 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.954 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.954 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.954 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-04-25T11:53:46.955 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-04-25T11:53:46.955 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-04-25T11:53:46.955 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.955 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-04-25T11:53:46.955 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-04-25T11:53:46.955 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.955 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.955 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.955 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-04-25T11:53:46.956 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.956 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.956 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.956 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T11:53:46.956 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.956 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.956 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.957 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-04-25T11:53:46.957 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-04-25T11:53:46.957 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.957 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.957 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.957 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-04-25T11:53:46.957 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.957 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.957 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.958 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T11:53:46.958 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.958 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.958 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.958 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-04-25T11:53:46.958 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-04-25T11:53:46.959 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.959 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.959 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.959 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.959 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.959 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.959 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-04-25T11:53:46.960 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.960 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.960 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.960 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-25T11:53:46.960 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-25T11:53:46.960 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:46.960 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.961 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.961 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-04-25T11:53:46.961 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-04-25T11:53:46.961 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.961 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.961 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T11:53:46.961 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.961 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.962 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.962 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.962 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.962 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.962 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.962 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.962 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.962 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.963 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.963 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.963 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.963 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.963 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.963 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.963 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.963 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.964 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.964 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.964 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.964 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.964 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.964 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.964 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.964 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.965 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.965 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.965 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.965 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.965 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.965 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.965 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.965 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.966 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.966 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.966 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.966 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.966 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.966 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.966 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.967 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.967 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.967 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.967 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.967 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.967 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.967 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.967 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.967 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.968 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.968 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.968 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.968 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.968 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.968 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.968 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.969 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.969 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.969 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.969 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.969 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.969 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.969 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.969 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.969 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.970 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.970 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.970 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.970 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.970 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-04-25T11:53:46.970 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-04-25T11:53:46.970 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.970 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.971 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.971 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.971 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.971 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.971 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-04-25T11:53:46.971 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.971 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.972 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.972 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-04-25T11:53:46.972 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-04-25T11:53:46.972 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.972 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.972 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.972 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-04-25T11:53:46.973 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.973 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.973 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.973 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-04-25T11:53:46.973 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.973 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.973 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.974 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-25T11:53:46.974 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-25T11:53:46.974 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.974 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.974 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.974 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-04-25T11:53:46.974 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.974 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.975 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.975 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:53:46.975 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.975 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.975 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-04-25T11:53:46.975 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-04-25T11:53:46.975 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:46.976 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.976 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.976 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-25T11:53:46.976 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-25T11:53:46.976 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-04-25T11:53:46.976 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.976 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T11:53:46.976 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.976 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.977 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.977 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-04-25T11:53:46.977 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-04-25T11:53:46.977 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.977 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.977 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.977 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.978 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.978 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.978 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.978 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-04-25T11:53:46.978 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.978 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.978 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.978 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.979 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-04-25T11:53:46.979 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-04-25T11:53:46.979 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.979 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.979 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:46.979 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-04-25T11:53:46.979 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.979 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.979 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.980 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T11:53:46.980 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.980 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.980 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.980 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.980 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.980 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.981 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-04-25T11:53:46.981 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.981 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.981 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.981 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.981 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.981 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.981 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.981 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.982 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.982 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.982 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.982 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.982 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.982 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.982 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.982 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.984 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.985 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.986 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.987 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.988 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-04-25T11:53:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:46.990 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.991 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.992 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.993 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:46.994 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:46.995 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:46.995 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.995 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.995 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:46.995 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.995 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.995 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:46.995 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:46.996 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:46.996 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:46.996 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:46.996 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:46.996 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:46.996 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:46.996 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.996 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.996 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.997 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.997 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:46.997 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:46.997 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:46.997 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:46.997 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:46.997 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:46.998 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:46.998 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:46.998 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:46.998 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:46.998 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:46.998 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:46.998 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:46.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.998 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:46.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:46.999 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:46.999 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-04-25T11:53:46.999 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:46.999 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:46.999 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:46.999 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-04-25T11:53:46.999 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-04-25T11:53:46.999 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.000 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.000 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.000 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-04-25T11:53:47.000 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.000 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.000 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.000 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T11:53:47.000 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.001 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.001 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.001 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-04-25T11:53:47.001 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-04-25T11:53:47.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.001 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.001 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.002 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:47.002 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-04-25T11:53:47.002 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.002 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.002 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T11:53:47.002 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.002 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.002 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.003 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.003 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.003 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.003 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.003 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.003 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.003 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.003 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.004 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.004 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.004 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.004 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.004 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.004 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.004 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.004 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.004 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.005 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.005 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.005 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.005 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.005 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.005 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.005 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.005 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.006 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.006 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.006 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.006 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.006 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.006 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.006 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.006 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.007 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.007 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.007 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.007 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.007 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.007 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.007 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.007 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.008 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.008 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.008 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.008 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.008 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.008 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.008 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.008 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.008 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.009 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.009 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.009 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.009 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.009 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.009 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.009 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.009 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.010 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.010 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.010 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.010 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.010 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.010 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.010 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.010 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.010 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.011 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.011 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.011 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.011 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.011 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-04-25T11:53:47.011 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.011 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.012 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.012 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-04-25T11:53:47.012 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-04-25T11:53:47.012 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.012 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.012 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.012 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-04-25T11:53:47.012 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.013 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.013 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.013 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-04-25T11:53:47.013 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T11:53:47.013 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.013 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.013 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.013 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.014 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.014 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.014 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.014 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.014 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.014 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.014 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.014 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.015 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.015 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.015 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.015 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.015 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.015 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.015 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.015 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.016 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.016 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.016 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.016 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.016 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.016 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.016 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.016 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.016 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.017 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.017 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.017 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.017 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.017 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.017 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.017 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.018 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.018 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.018 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.018 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.018 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.018 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.018 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.018 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.019 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.019 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.019 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.019 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.019 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.019 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.019 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.019 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.019 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.020 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.020 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.020 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.020 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.020 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.020 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.020 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.021 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.021 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.021 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.021 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.021 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.021 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.021 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.021 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.021 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.022 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-04-25T11:53:47.022 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-04-25T11:53:47.022 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.022 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.022 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.022 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.022 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.023 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.023 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.023 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-04-25T11:53:47.023 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T11:53:47.023 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.023 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.023 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.023 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-04-25T11:53:47.024 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-04-25T11:53:47.024 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.024 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.024 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.024 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-04-25T11:53:47.024 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.024 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.024 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.024 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-04-25T11:53:47.025 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.025 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.025 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.025 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-04-25T11:53:47.025 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-04-25T11:53:47.025 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.025 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.026 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.026 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-04-25T11:53:47.026 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.026 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.026 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.026 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T11:53:47.026 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.026 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.026 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.027 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-04-25T11:53:47.027 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-04-25T11:53:47.027 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.027 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.027 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.027 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.027 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.028 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.028 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.028 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-04-25T11:53:47.028 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.028 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.028 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.028 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-04-25T11:53:47.029 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-04-25T11:53:47.029 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.029 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.029 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.029 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-04-25T11:53:47.029 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.029 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.029 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.029 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T11:53:47.030 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T11:53:47.030 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.030 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.030 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.030 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.030 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.030 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-04-25T11:53:47.031 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.031 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.031 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.031 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.031 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.031 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.031 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.031 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.032 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.032 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.032 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.032 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.032 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.032 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.032 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.032 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.032 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.033 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.033 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.033 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.033 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.033 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.033 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.033 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.034 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.034 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.034 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.034 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.034 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.034 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.034 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.034 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.035 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.035 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.035 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.035 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.035 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.035 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.035 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.035 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.036 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.036 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.036 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.036 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.036 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.036 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.036 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.036 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.037 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.037 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.037 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.037 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.037 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.037 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.037 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.037 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.037 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.038 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.038 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.038 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.038 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.038 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.038 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.038 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.038 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.039 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.039 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.039 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.039 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.039 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.039 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.039 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.040 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.040 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-04-25T11:53:47.040 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.040 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.040 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.040 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.040 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.040 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.040 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.041 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.041 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.041 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.041 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.041 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.041 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.041 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.041 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.042 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.042 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.042 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.042 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.042 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.042 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.042 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.042 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.043 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.043 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.043 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.043 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.043 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.043 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.043 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.044 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.044 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.044 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.044 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.044 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.044 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.044 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.044 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.045 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.045 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.045 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.045 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.045 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.045 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.045 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.045 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.046 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.046 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.046 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.046 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.046 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.046 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.046 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.046 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.047 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.047 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.047 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.047 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.047 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.047 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.047 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.047 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.048 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.048 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.048 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.048 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.048 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.048 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.048 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.048 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.048 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.049 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.049 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-04-25T11:53:47.049 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.049 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.049 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.049 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.049 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.050 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.050 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.050 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.050 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.050 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.050 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.050 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.050 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.051 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.051 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.051 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.051 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.051 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.051 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.051 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.051 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.052 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.052 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.052 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.052 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.052 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.052 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.052 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.052 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.053 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.053 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.053 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.053 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.053 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.053 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.053 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.053 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.054 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.054 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.054 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.054 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.054 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.054 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.054 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.054 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.055 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.055 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.055 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.055 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.055 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.055 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.055 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.055 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.056 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.056 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.056 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.056 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.056 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.056 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.056 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.057 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.057 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.057 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.057 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.057 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.057 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.057 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.057 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.057 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.058 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.058 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.058 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.058 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-25T11:53:47.058 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.058 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.058 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.059 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-04-25T11:53:47.059 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-04-25T11:53:47.059 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.059 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.059 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.059 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.059 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.059 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.059 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.060 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T11:53:47.060 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.060 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.060 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.060 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.060 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.060 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.061 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.061 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.061 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.061 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.061 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.061 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.061 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.061 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.061 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.062 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.062 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.062 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.062 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.062 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.062 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.062 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.062 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.063 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.063 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.063 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.063 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.063 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.063 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.063 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.063 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.064 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.064 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.064 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.064 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.064 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.064 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.064 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.065 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.065 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.065 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.065 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.065 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.065 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.065 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.065 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.065 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.066 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.066 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.066 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.066 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.066 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.066 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.066 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.066 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.067 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.067 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.067 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.067 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.067 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.067 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.067 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.067 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.068 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.068 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.068 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.068 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.068 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.068 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.068 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-04-25T11:53:47.069 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-25T11:53:47.069 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.069 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.069 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.069 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.069 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.069 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.069 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.070 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-04-25T11:53:47.070 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-04-25T11:53:47.070 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-04-25T11:53:47.070 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.070 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-04-25T11:53:47.070 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.070 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.070 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.070 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-04-25T11:53:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.071 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.072 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.073 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.073 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.073 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.073 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.073 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.073 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.073 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.073 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.073 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.074 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.074 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.074 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.074 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.074 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.074 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.074 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.075 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.075 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.075 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.075 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.075 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.075 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.075 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.075 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.076 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.076 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.076 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.076 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.076 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.076 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.076 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.076 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.077 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.077 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.077 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.077 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.077 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.077 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.077 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.077 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.077 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.078 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.078 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.078 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.078 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.078 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.078 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.078 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.079 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.079 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.079 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.079 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.079 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.079 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.079 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.079 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.080 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.080 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.080 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.080 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-04-25T11:53:47.080 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.080 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.080 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.081 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.081 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.081 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.081 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.081 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.081 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.081 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.081 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.081 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.082 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.082 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.082 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.082 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.082 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.082 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.082 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.082 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.083 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.083 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.083 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.083 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.083 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.083 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.083 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.084 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.084 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.084 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.084 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.084 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.084 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.084 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.084 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.084 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.085 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.085 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.085 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.085 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.085 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.085 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.085 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.086 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.086 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.086 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.086 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.086 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.086 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.086 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.086 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.087 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.087 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.087 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.087 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.087 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.087 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.087 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.087 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.088 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.088 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.088 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.088 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.088 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.088 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.088 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.088 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.089 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.089 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.089 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.089 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.089 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.089 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-04-25T11:53:47.089 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.089 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.090 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.090 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.090 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.090 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.090 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.090 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.090 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.090 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.091 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.091 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.091 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.091 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.091 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.091 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.091 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.091 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.091 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.092 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.092 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.092 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.092 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.092 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.092 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.092 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.093 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.093 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.093 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.093 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.093 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.093 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.093 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.093 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.093 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.094 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.094 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.094 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.094 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.094 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.094 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.094 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.094 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.095 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.095 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.095 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.095 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.095 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.095 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.095 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.095 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.095 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.096 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.096 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.096 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.096 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.096 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.096 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.096 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.096 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.097 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.097 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.097 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.097 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.097 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.097 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.097 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.097 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.098 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.098 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.098 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-25T11:53:47.098 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-25T11:53:47.098 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:47.098 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.098 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.099 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-25T11:53:47.099 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.099 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.099 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.099 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.099 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.099 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.099 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.100 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-04-25T11:53:47.100 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-04-25T11:53:47.100 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:47.100 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.100 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.100 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-25T11:53:47.100 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-04-25T11:53:47.101 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.101 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.101 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-04-25T11:53:47.101 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T11:53:47.101 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.101 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.101 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.101 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.102 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.102 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-04-25T11:53:47.102 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.102 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.102 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.102 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-04-25T11:53:47.103 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-04-25T11:53:47.103 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.103 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.103 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.103 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.103 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.103 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.103 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.103 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T11:53:47.104 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.104 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.104 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.104 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-25T11:53:47.104 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-25T11:53:47.104 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.104 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.105 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.105 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:47.105 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.105 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.105 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.105 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.105 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.105 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.105 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.106 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.106 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.106 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.106 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.106 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.106 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.106 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.107 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.107 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.107 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.107 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.107 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.107 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.107 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.107 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.107 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.108 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.108 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.108 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.108 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.108 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.108 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.108 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.108 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.109 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.109 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.109 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.109 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.109 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.109 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.109 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.109 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.110 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.110 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.110 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.110 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.110 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.110 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.110 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.110 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.111 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.111 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.111 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.111 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.111 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.111 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.111 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.111 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.112 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.112 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.112 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.112 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.112 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.112 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.112 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.112 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.113 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.113 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.113 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.113 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.113 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.113 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.113 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.113 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.113 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.114 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.114 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-04-25T11:53:47.114 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-04-25T11:53:47.114 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.114 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.114 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.114 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-04-25T11:53:47.115 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.115 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.115 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.115 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-04-25T11:53:47.115 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.115 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.115 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.115 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-25T11:53:47.116 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-25T11:53:47.116 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:47.116 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.116 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.116 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-25T11:53:47.116 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.116 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.116 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.117 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-04-25T11:53:47.117 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.117 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.117 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.117 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.117 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.117 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.118 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-04-25T11:53:47.118 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.118 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.118 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.118 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-04-25T11:53:47.118 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-04-25T11:53:47.118 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.118 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.118 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.119 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:47.119 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.119 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.119 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.119 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.119 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.119 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.119 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.120 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-04-25T11:53:47.120 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-04-25T11:53:47.120 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.120 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.120 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.120 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:47.120 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.120 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.121 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.121 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T11:53:47.121 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T11:53:47.121 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.121 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.121 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.121 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.122 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.122 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.122 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.122 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.122 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.122 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.122 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.122 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.123 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.123 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.123 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.123 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.123 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.123 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.123 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.123 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.123 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.124 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.124 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.124 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.124 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.124 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.124 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.124 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.125 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.125 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.125 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.125 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.125 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.125 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.125 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.125 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.126 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.126 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.126 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.126 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.126 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.126 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.126 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.126 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.127 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.127 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.127 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.127 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.127 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.127 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.127 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.127 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.127 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.128 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.128 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.128 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.128 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.128 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.128 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.128 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.129 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.129 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.129 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.129 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.129 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.129 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.129 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.129 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.130 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-04-25T11:53:47.130 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-04-25T11:53:47.130 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.130 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.130 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.130 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.130 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.130 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.131 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.131 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.131 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.131 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.131 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.131 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T11:53:47.131 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T11:53:47.132 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.132 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.132 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.132 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-04-25T11:53:47.132 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.132 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.132 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.132 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.132 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.133 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.133 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.133 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-04-25T11:53:47.133 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-04-25T11:53:47.133 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:47.133 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.133 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.134 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-04-25T11:53:47.134 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.134 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.134 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.134 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.134 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.134 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.134 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.135 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-25T11:53:47.135 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-25T11:53:47.135 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.135 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.135 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.135 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-25T11:53:47.135 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.135 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.136 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.136 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T11:53:47.136 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.136 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.136 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.136 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-25T11:53:47.136 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-25T11:53:47.137 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.137 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.137 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.137 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-04-25T11:53:47.137 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.137 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.137 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.137 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T11:53:47.137 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.138 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.138 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.138 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-04-25T11:53:47.138 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-04-25T11:53:47.138 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.138 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.138 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.139 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-04-25T11:53:47.139 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.139 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.139 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.139 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.139 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.139 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.139 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.140 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-25T11:53:47.140 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-25T11:53:47.140 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.140 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.140 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.140 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-04-25T11:53:47.140 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.140 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.141 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.141 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-04-25T11:53:47.141 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-04-25T11:53:47.141 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.141 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.141 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.141 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.141 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.142 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-25T11:53:47.142 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-25T11:53:47.142 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.142 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.142 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.142 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-25T11:53:47.142 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-25T11:53:47.143 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-25T11:53:47.143 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.143 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.143 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.143 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.143 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.143 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.143 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.144 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.144 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-04-25T11:53:47.144 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.144 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.144 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.144 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.144 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.144 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.145 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.145 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.145 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.145 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.145 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.145 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.145 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.145 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.145 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.146 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.146 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.146 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.146 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.146 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.146 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.146 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.146 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.147 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.147 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.147 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.147 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.147 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.147 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.147 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.147 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.148 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.148 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.148 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.148 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.148 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.148 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.148 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.148 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.148 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.149 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.149 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.149 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.149 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.149 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.149 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.149 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.149 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.150 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.150 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.150 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.150 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.150 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.150 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.150 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.150 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.150 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.151 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.151 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.151 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.151 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.151 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.151 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.151 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.151 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.152 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.152 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.152 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.152 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.152 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.152 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.152 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-04-25T11:53:47.152 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-04-25T11:53:47.153 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.153 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.153 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.153 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-25T11:53:47.153 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.153 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.153 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.153 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.153 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.154 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.154 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.154 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-04-25T11:53:47.154 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T11:53:47.154 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.154 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.154 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.155 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.155 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.155 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.155 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.155 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.155 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.155 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.155 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.156 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-04-25T11:53:47.156 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-04-25T11:53:47.156 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.156 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.156 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.156 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.156 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.156 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.157 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.157 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.157 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.157 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.157 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.157 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.157 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.158 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-04-25T11:53:47.158 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.158 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.158 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.158 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-04-25T11:53:47.158 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-04-25T11:53:47.158 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.158 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.159 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.159 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:47.159 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.159 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.159 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.159 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.159 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.159 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.160 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.160 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-25T11:53:47.160 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-25T11:53:47.160 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.160 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.160 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.160 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.160 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.161 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.161 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.161 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.161 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.161 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.161 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.161 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.162 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.162 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.162 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.162 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.162 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.162 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.162 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.162 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.162 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.163 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.163 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.163 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.163 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.163 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.163 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.163 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.163 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.164 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.164 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.164 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.164 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.164 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.164 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.164 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.164 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.165 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.165 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.165 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.165 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.165 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.165 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.165 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.166 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.166 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.166 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.166 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.166 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.166 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.166 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.166 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.167 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.167 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.167 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.167 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.167 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.167 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.167 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.167 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.167 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.168 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.168 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.168 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.168 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.168 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.168 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.168 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.168 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.169 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.169 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.169 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.169 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.169 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.169 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.169 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.169 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.170 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-25T11:53:47.170 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-25T11:53:47.170 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.170 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.170 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.170 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.171 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.171 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.171 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.171 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.171 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.171 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.171 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.171 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-25T11:53:47.172 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-25T11:53:47.172 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.172 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.172 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.172 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.172 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.172 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.172 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.173 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.173 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.173 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.173 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.173 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.173 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.173 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.173 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-04-25T11:53:47.174 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.174 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.174 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.174 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.174 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.174 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.174 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.174 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.175 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.175 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.175 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.175 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.175 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.175 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.175 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.175 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.175 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.176 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.176 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.176 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.176 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.176 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.176 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.176 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.177 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.177 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.177 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.177 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.177 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.177 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.177 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.177 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.177 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.178 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.178 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.178 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.178 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.178 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.178 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.178 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.179 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.179 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.179 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.179 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.179 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.179 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.179 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.179 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.179 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.180 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.180 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.180 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.180 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.180 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.180 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.180 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.180 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.181 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.181 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.181 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.181 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.181 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.181 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.181 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.181 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.182 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.182 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.182 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.182 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.182 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.182 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.182 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.182 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.183 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-04-25T11:53:47.183 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.183 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.183 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.183 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.183 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.183 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.183 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.184 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.184 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.184 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.184 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.184 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.184 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.184 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.184 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.184 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.185 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.185 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.185 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.185 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.185 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.185 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.185 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.186 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.186 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.186 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.186 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.186 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.186 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.187 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.187 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.187 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.187 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.187 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.187 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.187 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.188 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.188 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.188 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.188 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.188 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.188 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.188 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.189 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.189 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.189 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.189 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.189 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.189 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.189 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.189 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.190 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.190 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.190 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.190 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.190 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.190 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.190 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.191 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.191 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.191 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.191 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.191 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.191 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.191 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.192 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.192 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.192 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.192 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.192 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-04-25T11:53:47.192 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-04-25T11:53:47.192 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.193 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.193 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.193 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-04-25T11:53:47.193 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.193 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.193 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.193 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T11:53:47.193 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.193 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.194 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.194 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.194 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.194 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.194 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-04-25T11:53:47.194 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.194 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.195 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.195 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.195 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.195 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.195 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.195 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.195 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.195 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.196 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.196 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.196 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.196 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.196 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.196 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.196 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.196 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.196 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.197 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.197 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.197 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.197 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.197 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.197 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.197 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.198 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.198 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.198 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.198 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.198 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.198 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.198 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.198 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.198 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.199 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.199 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.199 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.199 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.199 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.199 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.199 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.199 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.200 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.200 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.200 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.200 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.200 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.200 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.200 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.200 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.200 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.201 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.201 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.201 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.201 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.201 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.201 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.201 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.201 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.202 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.202 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.202 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.202 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.202 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.202 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.202 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.202 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.203 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.203 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.203 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-04-25T11:53:47.203 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-04-25T11:53:47.203 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.203 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.203 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.203 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.204 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.204 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.204 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.204 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.204 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.204 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.204 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.205 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-04-25T11:53:47.205 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-04-25T11:53:47.205 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.205 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.205 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.205 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-04-25T11:53:47.205 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.205 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.206 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.206 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.206 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.206 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.206 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.206 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-04-25T11:53:47.206 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-04-25T11:53:47.206 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.207 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.207 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.207 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.207 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.207 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.207 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.207 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.207 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.208 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.208 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.208 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-04-25T11:53:47.208 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-04-25T11:53:47.208 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.208 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.208 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.208 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.209 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.209 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.209 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.209 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.209 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.209 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.209 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.210 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-04-25T11:53:47.210 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-04-25T11:53:47.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-25T11:53:47.210 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.210 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.210 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.210 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.210 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.210 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.211 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.211 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.211 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.211 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.211 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-04-25T11:53:47.211 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-04-25T11:53:47.211 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.212 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.212 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.212 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.212 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.212 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.212 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.212 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.212 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.212 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.213 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.213 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-04-25T11:53:47.213 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-04-25T11:53:47.213 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.213 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.213 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.213 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.214 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.214 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.214 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.214 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.214 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.214 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.214 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.215 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-04-25T11:53:47.215 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-04-25T11:53:47.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.215 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.215 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:47.215 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.215 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.215 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.216 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.216 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.216 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.216 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.216 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-04-25T11:53:47.216 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-04-25T11:53:47.216 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.216 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.217 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.217 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.217 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-25T11:53:47.217 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-04-25T11:53:47.217 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.217 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.217 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.217 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.218 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.218 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-04-25T11:53:47.218 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-04-25T11:53:47.218 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.218 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.218 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.218 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.219 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.219 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.219 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.219 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.219 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.219 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.219 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.219 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-04-25T11:53:47.220 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-04-25T11:53:47.220 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.220 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.220 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.220 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.220 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.220 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.220 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.220 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.221 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.221 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.221 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.221 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.221 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-04-25T11:53:47.221 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-04-25T11:53:47.221 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.221 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.222 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.222 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.222 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.222 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.222 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.222 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.222 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.222 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.223 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.223 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.223 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.223 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.223 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-04-25T11:53:47.223 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.223 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.223 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.224 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-04-25T11:53:47.224 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-04-25T11:53:47.224 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.224 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.224 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.224 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.224 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.224 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.225 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.225 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T11:53:47.225 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.225 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.225 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.225 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-04-25T11:53:47.225 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-04-25T11:53:47.225 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.226 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.226 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.226 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.226 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.226 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.226 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.226 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-04-25T11:53:47.226 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.227 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.227 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.227 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.227 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.227 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.227 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.227 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.228 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.228 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.228 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.228 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.228 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.228 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.228 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.228 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.228 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.229 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.229 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.229 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.229 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.229 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.229 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.229 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.229 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.230 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.230 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.230 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.230 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.230 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.230 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.230 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.230 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.231 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.231 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.231 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.231 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.231 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.231 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.231 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.231 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.232 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.232 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.232 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.232 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.232 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.232 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.232 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.232 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.233 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.233 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.233 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.233 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.233 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.233 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.233 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.233 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.234 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.234 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.234 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.234 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.234 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.234 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.234 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.234 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.235 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.235 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.235 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.235 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.235 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.235 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.236 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-04-25T11:53:47.236 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.236 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.236 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.236 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.236 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.236 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.237 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.237 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.237 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.237 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.237 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.237 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.237 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.237 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.238 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.238 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.238 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.238 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.238 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.238 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.238 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.238 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.239 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.239 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.239 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.239 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.239 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.239 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.239 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.239 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.240 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.240 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.240 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.240 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.240 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.240 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.240 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.240 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.240 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.241 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.241 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.241 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.241 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.241 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.241 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.241 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.241 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.242 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.242 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.242 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.242 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.242 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.242 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.242 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.242 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.243 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.243 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.243 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.243 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.243 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.243 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.243 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.243 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.244 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.244 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.244 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.244 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.244 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.244 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.245 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.245 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-04-25T11:53:47.245 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.245 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.245 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.245 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.245 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.246 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.246 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.246 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.246 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.246 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.246 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.246 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.246 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.246 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.247 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.247 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.247 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.247 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.247 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.247 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.247 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.247 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.248 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.248 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.248 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.248 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.248 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.248 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.248 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.248 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.249 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.249 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.249 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.249 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.249 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.249 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.249 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.250 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.250 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.250 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.250 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.250 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.250 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.250 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.250 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.250 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.251 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.251 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.251 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.251 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.251 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.251 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.251 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.251 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.252 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.252 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.252 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.252 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.252 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.252 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.252 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.252 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.253 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.253 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.253 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.253 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.253 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.253 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.253 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.253 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.253 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.254 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.254 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-04-25T11:53:47.254 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.254 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.254 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.254 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-04-25T11:53:47.254 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-04-25T11:53:47.254 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.255 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.255 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.255 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T11:53:47.255 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.255 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.255 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.255 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.255 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.255 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.256 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.256 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.256 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-25T11:53:47.256 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-25T11:53:47.256 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:47.256 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.256 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.256 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-04-25T11:53:47.257 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.257 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.257 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.257 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.257 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.257 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.257 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.257 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.258 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.258 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.258 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.258 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.258 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.258 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.258 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.259 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.259 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.259 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.259 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.259 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.259 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.259 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.259 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.259 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.260 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.260 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.260 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.260 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.260 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.260 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.260 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.261 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.261 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.261 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.261 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.261 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.261 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.261 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.261 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.262 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.262 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.262 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.262 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.262 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.262 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.262 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.262 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.263 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.263 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.263 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.263 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.263 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.263 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.263 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.263 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.263 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.264 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.264 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.264 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.264 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.264 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.264 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.264 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.265 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.265 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.265 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.265 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.265 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.265 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.265 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.265 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.265 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.266 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.266 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.266 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.266 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-04-25T11:53:47.266 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.266 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.266 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.267 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-04-25T11:53:47.267 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-04-25T11:53:47.267 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.267 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.267 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.267 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:47.267 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.267 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.267 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.268 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T11:53:47.268 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.268 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.268 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.268 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-04-25T11:53:47.268 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-04-25T11:53:47.268 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.269 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.269 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.269 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:47.269 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.269 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.269 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.269 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T11:53:47.269 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.269 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.270 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.270 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-04-25T11:53:47.270 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-04-25T11:53:47.270 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.270 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.270 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.270 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:47.271 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.271 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.271 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.271 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T11:53:47.271 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.271 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.271 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.272 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-04-25T11:53:47.272 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-04-25T11:53:47.272 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.272 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.272 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.272 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.272 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.272 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.272 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.273 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T11:53:47.273 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.273 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.273 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.273 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-04-25T11:53:47.273 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-04-25T11:53:47.273 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.274 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.274 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.274 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.274 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.274 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.274 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.274 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T11:53:47.274 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.274 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.275 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.275 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-04-25T11:53:47.275 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-04-25T11:53:47.275 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.275 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.275 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.275 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.276 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.276 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.276 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.276 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.276 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.276 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.276 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.276 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-04-25T11:53:47.277 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-04-25T11:53:47.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.277 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.277 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.277 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.277 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.277 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.278 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.278 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.278 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.278 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.278 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-04-25T11:53:47.278 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-04-25T11:53:47.278 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.279 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.279 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.279 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T11:53:47.279 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.279 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.279 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.279 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.279 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.279 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.280 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.280 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-25T11:53:47.280 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-25T11:53:47.280 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.280 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.280 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.280 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.281 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.281 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.281 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.281 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.281 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.281 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.281 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.282 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-25T11:53:47.282 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-25T11:53:47.282 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.282 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.282 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.282 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-04-25T11:53:47.282 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-04-25T11:53:47.282 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.283 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.283 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.283 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.283 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.283 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.283 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-04-25T11:53:47.283 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-04-25T11:53:47.283 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.284 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.284 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.284 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-04-25T11:53:47.284 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.284 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.284 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.284 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.284 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.285 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.285 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.285 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-04-25T11:53:47.285 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-04-25T11:53:47.285 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.285 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.285 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.286 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.286 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.286 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.286 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.286 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.286 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.286 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.286 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.287 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-04-25T11:53:47.287 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-04-25T11:53:47.287 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.287 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.287 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.287 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.287 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.288 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.288 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.288 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.288 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.288 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.288 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.288 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.288 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.288 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.289 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.289 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.289 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.289 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.289 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.289 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.289 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.289 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.290 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.290 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.290 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.290 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.290 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.290 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.290 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.290 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.290 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.291 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.291 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.291 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.291 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.291 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.291 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.291 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.292 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.292 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.292 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.292 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.292 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.292 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.292 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.292 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.293 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.293 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.293 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.293 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.293 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.293 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.293 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.293 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.293 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.294 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.294 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.294 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.294 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.294 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.294 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.294 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.294 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.295 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.295 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.295 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.295 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.295 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.295 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.295 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.296 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.296 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.296 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.296 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.296 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.296 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.296 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.296 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.296 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.297 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-04-25T11:53:47.297 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-04-25T11:53:47.297 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.297 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.297 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.297 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.297 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.297 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.297 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.298 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.298 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.298 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.298 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.298 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.298 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-04-25T11:53:47.298 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-04-25T11:53:47.298 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.299 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.299 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.299 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.299 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.299 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.299 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.299 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.299 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.300 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.300 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.300 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-04-25T11:53:47.300 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-04-25T11:53:47.300 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.300 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.300 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.300 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T11:53:47.301 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.301 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.301 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.301 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.301 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.301 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.301 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.301 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.302 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.302 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.302 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.302 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-04-25T11:53:47.302 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.302 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.302 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.302 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.302 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.303 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.303 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.303 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.303 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.303 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.303 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.303 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.303 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.304 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.304 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.304 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.304 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.304 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.304 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.304 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.304 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.304 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.305 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.305 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.305 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.305 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.305 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.305 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.305 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.306 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.306 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.306 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.306 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.306 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.306 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.306 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.306 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.306 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.307 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.307 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.310 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.310 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.310 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.310 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.310 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.310 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.311 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.311 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.311 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.311 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.311 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.311 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.311 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.311 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.311 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.312 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.312 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.312 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.312 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.312 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.312 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.312 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.312 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.313 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.313 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.313 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.313 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.313 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.313 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.313 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.313 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.313 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.314 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.314 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.314 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.314 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-04-25T11:53:47.314 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.314 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.314 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.314 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.315 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.315 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.315 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.315 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.315 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.315 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.315 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.316 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.316 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.316 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.316 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.316 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.316 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.316 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.316 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.316 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.317 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.317 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.317 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.317 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.317 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.317 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.317 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.318 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.318 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.318 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.318 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.318 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.318 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.318 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.318 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.319 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.319 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.319 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.319 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.319 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.319 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.319 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.319 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.320 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.320 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.320 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.320 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.320 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.320 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.320 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.320 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.320 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.321 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.321 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.321 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.321 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.321 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.321 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.321 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.322 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.322 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.322 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.322 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.322 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.322 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.322 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.322 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.323 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-04-25T11:53:47.323 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-04-25T11:53:47.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.323 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.323 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.323 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-04-25T11:53:47.323 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.323 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.324 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.324 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T11:53:47.324 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.324 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.324 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.324 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-04-25T11:53:47.324 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-04-25T11:53:47.325 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.325 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.325 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.325 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.325 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.325 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.325 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.325 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-04-25T11:53:47.325 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.326 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.326 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.326 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.326 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-04-25T11:53:47.326 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-04-25T11:53:47.326 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.326 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.326 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.326 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-25T11:53:47.327 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.327 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.327 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.327 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T11:53:47.327 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.327 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.327 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.328 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_no_wait": { 2024-04-25T11:53:47.328 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T11:53:47.328 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.328 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.328 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.328 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-25T11:53:47.328 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.328 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.328 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.329 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T11:53:47.329 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.329 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.329 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.329 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.329 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.329 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.329 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.330 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-04-25T11:53:47.330 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-25T11:53:47.330 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-25T11:53:47.330 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-25T11:53:47.330 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-04-25T11:53:47.330 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-04-25T11:53:47.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-25T11:53:47.330 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.330 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.331 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-04-25T11:53:47.331 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.331 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.331 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.331 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.331 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.331 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.331 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.332 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-04-25T11:53:47.332 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-04-25T11:53:47.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.332 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.332 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.332 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.332 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.333 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.333 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.333 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.333 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.333 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.333 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-25T11:53:47.333 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-25T11:53:47.334 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-25T11:53:47.334 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.334 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.334 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-25T11:53:47.334 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.334 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.334 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.334 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.334 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.335 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.335 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.335 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-25T11:53:47.335 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-25T11:53:47.335 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.335 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.335 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.336 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.336 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.336 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.336 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.336 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.336 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.336 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.336 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.336 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.337 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.337 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.337 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.337 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.337 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.337 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.337 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-25T11:53:47.338 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-25T11:53:47.338 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.338 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.338 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.338 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.338 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.338 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.338 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.338 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.339 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.339 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.339 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.339 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-25T11:53:47.339 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-25T11:53:47.339 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.339 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.340 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.340 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-25T11:53:47.340 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.340 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.340 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-25T11:53:47.340 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-25T11:53:47.340 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-25T11:53:47.340 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-25T11:53:47.340 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-25T11:53:47.341 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-25T11:53:47.341 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-25T11:53:47.341 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.341 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.341 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.341 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.341 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.341 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-25T11:53:47.342 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-25T11:53:47.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-25T11:53:47.342 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.342 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-25T11:53:47.342 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-25T11:53:47.342 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.342 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.342 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.343 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.343 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.343 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.343 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.343 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-04-25T11:53:47.343 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-04-25T11:53:47.343 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.343 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.344 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.344 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-25T11:53:47.344 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.344 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.344 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.344 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.344 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.344 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.345 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.345 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-04-25T11:53:47.345 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-04-25T11:53:47.345 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-25T11:53:47.345 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.345 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.345 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-04-25T11:53:47.345 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.346 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.346 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.346 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.346 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.346 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.346 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.346 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-04-25T11:53:47.347 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-04-25T11:53:47.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-25T11:53:47.347 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-25T11:53:47.347 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-25T11:53:47.347 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T11:53:47.347 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-25T11:53:47.347 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-25T11:53:47.347 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-25T11:53:47.348 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-25T11:53:47.348 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-25T11:53:47.348 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-25T11:53:47.348 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-25T11:53:47.348 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.348 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.348 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.348 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.348 INFO:teuthology.orchestra.run.smithi083.stdout: "services": { 2024-04-25T11:53:47.349 INFO:teuthology.orchestra.run.smithi083.stdout: "dashboard": "https://172.21.15.83:8443/" 2024-04-25T11:53:47.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.349 INFO:teuthology.orchestra.run.smithi083.stdout: "always_on_modules": { 2024-04-25T11:53:47.349 INFO:teuthology.orchestra.run.smithi083.stdout: "octopus": [ 2024-04-25T11:53:47.349 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-04-25T11:53:47.349 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-04-25T11:53:47.349 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-04-25T11:53:47.349 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-04-25T11:53:47.349 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-04-25T11:53:47.350 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-04-25T11:53:47.350 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-04-25T11:53:47.350 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-04-25T11:53:47.350 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-04-25T11:53:47.350 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-04-25T11:53:47.350 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.350 INFO:teuthology.orchestra.run.smithi083.stdout: "pacific": [ 2024-04-25T11:53:47.350 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-04-25T11:53:47.351 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-04-25T11:53:47.351 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-04-25T11:53:47.351 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-04-25T11:53:47.351 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-04-25T11:53:47.351 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-04-25T11:53:47.351 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-04-25T11:53:47.351 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-04-25T11:53:47.351 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-04-25T11:53:47.351 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-04-25T11:53:47.352 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.352 INFO:teuthology.orchestra.run.smithi083.stdout: "quincy": [ 2024-04-25T11:53:47.352 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-04-25T11:53:47.352 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-04-25T11:53:47.352 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-04-25T11:53:47.352 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-04-25T11:53:47.352 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-04-25T11:53:47.352 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-04-25T11:53:47.353 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-04-25T11:53:47.353 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-04-25T11:53:47.353 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-04-25T11:53:47.353 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-04-25T11:53:47.353 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.353 INFO:teuthology.orchestra.run.smithi083.stdout: "reef": [ 2024-04-25T11:53:47.353 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-04-25T11:53:47.353 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-04-25T11:53:47.353 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-04-25T11:53:47.354 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-04-25T11:53:47.354 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-04-25T11:53:47.354 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-04-25T11:53:47.354 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-04-25T11:53:47.354 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-04-25T11:53:47.354 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-04-25T11:53:47.354 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-04-25T11:53:47.354 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-25T11:53:47.355 INFO:teuthology.orchestra.run.smithi083.stdout: "squid": [ 2024-04-25T11:53:47.355 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-04-25T11:53:47.355 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-04-25T11:53:47.355 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-04-25T11:53:47.355 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-04-25T11:53:47.355 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-04-25T11:53:47.355 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-04-25T11:53:47.355 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-04-25T11:53:47.355 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-04-25T11:53:47.356 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-04-25T11:53:47.356 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-04-25T11:53:47.356 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T11:53:47.356 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.356 INFO:teuthology.orchestra.run.smithi083.stdout: "last_failure_osd_epoch": 89, 2024-04-25T11:53:47.356 INFO:teuthology.orchestra.run.smithi083.stdout: "active_clients": [ 2024-04-25T11:53:47.356 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.356 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-04-25T11:53:47.357 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-25T11:53:47.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.357 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-25T11:53:47.357 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-25T11:53:47.357 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2477057844 2024-04-25T11:53:47.357 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.357 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T11:53:47.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.357 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.358 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "libcephsqlite", 2024-04-25T11:53:47.358 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-25T11:53:47.358 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.358 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-25T11:53:47.358 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-25T11:53:47.358 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 796197009 2024-04-25T11:53:47.358 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.358 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T11:53:47.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.359 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-04-25T11:53:47.359 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-25T11:53:47.359 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.359 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-25T11:53:47.359 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-25T11:53:47.359 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 3949111144 2024-04-25T11:53:47.359 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.360 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T11:53:47.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.360 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-04-25T11:53:47.360 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-25T11:53:47.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.360 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-25T11:53:47.360 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-25T11:53:47.361 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 327612543 2024-04-25T11:53:47.361 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.361 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T11:53:47.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.361 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-04-25T11:53:47.361 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-25T11:53:47.361 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.361 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-25T11:53:47.362 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-25T11:53:47.362 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2866790759 2024-04-25T11:53:47.362 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.362 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T11:53:47.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:47.362 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.362 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-04-25T11:53:47.362 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-25T11:53:47.363 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-25T11:53:47.363 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-25T11:53:47.363 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-25T11:53:47.363 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2001314972 2024-04-25T11:53:47.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.363 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T11:53:47.363 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:47.363 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-25T11:53:47.363 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:53:47.373 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T11:53:47.373 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T11:53:47.374 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:45 smithi083 ceph-mon[29330]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:47.374 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:45 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1807452094' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T11:53:47.620 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:53:48.024 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:47 smithi083 ceph-mon[29330]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T11:53:48.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:47 smithi094 ceph-mon[33032]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T11:53:48.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:47 smithi111 ceph-mon[32784]: pgmap v314: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T11:53:49.372 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:53:49.373 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:53:49.373 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1438, 2024-04-25T11:53:49.373 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1440, 2024-04-25T11:53:49.373 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 431437, 2024-04-25T11:53:49.373 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 457285, 2024-04-25T11:53:49.374 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 6, 2024-04-25T11:53:49.374 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-04-25T11:53:49.374 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.135237376, 2024-04-25T11:53:49.374 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.047134987, 2024-04-25T11:53:49.374 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.033398556, 2024-04-25T11:53:49.374 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.040691908, 2024-04-25T11:53:49.374 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:53:49.375 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1438, 2024-04-25T11:53:49.375 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.055397346, 2024-04-25T11:53:49.375 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000038523 2024-04-25T11:53:49.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.375 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:53:49.375 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.375 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.376 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.376 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.376 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:53:49.376 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:53:49.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.377 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:53:49.377 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 613, 2024-04-25T11:53:49.377 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 761, 2024-04-25T11:53:49.377 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 151278, 2024-04-25T11:53:49.377 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 624782, 2024-04-25T11:53:49.377 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 7, 2024-04-25T11:53:49.378 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-04-25T11:53:49.378 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.071420710, 2024-04-25T11:53:49.378 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.025192526, 2024-04-25T11:53:49.378 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.016008788, 2024-04-25T11:53:49.378 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.023324022, 2024-04-25T11:53:49.378 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:53:49.378 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 758, 2024-04-25T11:53:49.378 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.030450321, 2024-04-25T11:53:49.379 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000040171 2024-04-25T11:53:49.379 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.379 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:53:49.379 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.379 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.379 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.380 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:53:49.380 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 430416 2024-04-25T11:53:49.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.380 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:53:49.380 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 231, 2024-04-25T11:53:49.380 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 217, 2024-04-25T11:53:49.381 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1618492, 2024-04-25T11:53:49.381 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 125038, 2024-04-25T11:53:49.381 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 3, 2024-04-25T11:53:49.381 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-04-25T11:53:49.381 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.040726000, 2024-04-25T11:53:49.381 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.014167757, 2024-04-25T11:53:49.381 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.030144389, 2024-04-25T11:53:49.381 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.004222308, 2024-04-25T11:53:49.382 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:53:49.382 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 217, 2024-04-25T11:53:49.382 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.012410023, 2024-04-25T11:53:49.382 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000057189 2024-04-25T11:53:49.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.383 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:53:49.384 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.384 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.384 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.385 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1618224, 2024-04-25T11:53:49.385 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 124720 2024-04-25T11:53:49.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.386 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-25T11:53:49.387 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-25T11:53:49.387 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-25T11:53:49.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.387 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-25T11:53:49.387 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-25T11:53:49.389 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-25T11:53:49.389 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3351, 2024-04-25T11:53:49.390 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.090750317, 2024-04-25T11:53:49.390 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000027081 2024-04-25T11:53:49.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:49.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.390 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-25T11:53:49.390 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-25T11:53:49.390 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-25T11:53:49.390 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.391 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.391 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.391 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:49.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.391 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-25T11:53:49.391 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 0, 2024-04-25T11:53:49.392 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 0, 2024-04-25T11:53:49.392 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-25T11:53:49.392 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.392 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.393 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.393 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-25T11:53:49.393 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-04-25T11:53:49.393 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-25T11:53:49.393 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-25T11:53:49.393 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 0, 2024-04-25T11:53:49.393 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-04-25T11:53:49.394 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-04-25T11:53:49.394 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 0, 2024-04-25T11:53:49.394 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 0, 2024-04-25T11:53:49.394 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 0, 2024-04-25T11:53:49.394 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-04-25T11:53:49.394 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 0, 2024-04-25T11:53:49.395 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-04-25T11:53:49.395 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 0, 2024-04-25T11:53:49.395 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 0, 2024-04-25T11:53:49.396 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 1, 2024-04-25T11:53:49.396 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 0, 2024-04-25T11:53:49.396 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 0, 2024-04-25T11:53:49.396 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-04-25T11:53:49.397 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-04-25T11:53:49.397 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:53:49.397 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-25T11:53:49.398 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-04-25T11:53:49.398 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-25T11:53:49.398 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-25T11:53:49.398 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-04-25T11:53:49.398 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-04-25T11:53:49.398 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-04-25T11:53:49.399 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-04-25T11:53:49.399 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-25T11:53:49.399 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:53:49.399 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-25T11:53:49.399 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 0, 2024-04-25T11:53:49.399 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 0, 2024-04-25T11:53:49.399 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-04-25T11:53:49.399 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:53:49.400 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-25T11:53:49.400 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:53:49.400 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:53:49.400 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:53:49.402 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:53:49.402 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:53:49.402 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-04-25T11:53:49.402 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:53:49.402 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:53:49.403 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:53:49.403 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:53:49.403 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:53:49.403 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:53:49.403 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 0, 2024-04-25T11:53:49.404 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:53:49.404 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 0, 2024-04-25T11:53:49.404 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 0 2024-04-25T11:53:49.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.404 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-25T11:53:49.404 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-25T11:53:49.405 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-25T11:53:49.405 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-25T11:53:49.405 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-04-25T11:53:49.405 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-04-25T11:53:49.406 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-04-25T11:53:49.406 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-25T11:53:49.406 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:53:49.406 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 0, 2024-04-25T11:53:49.407 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 0, 2024-04-25T11:53:49.407 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-04-25T11:53:49.407 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-25T11:53:49.407 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-25T11:53:49.408 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:53:49.408 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:53:49.408 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-25T11:53:49.408 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-25T11:53:49.409 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:53:49.409 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:53:49.409 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:53:49.409 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-04-25T11:53:49.410 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-25T11:53:49.410 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:53:49.410 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-25T11:53:49.410 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-25T11:53:49.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.410 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-25T11:53:49.410 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-25T11:53:49.411 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 0, 2024-04-25T11:53:49.411 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-04-25T11:53:49.411 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-04-25T11:53:49.411 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 6, 2024-04-25T11:53:49.411 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-25T11:53:49.411 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-04-25T11:53:49.411 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 0, 2024-04-25T11:53:49.412 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-04-25T11:53:49.412 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-04-25T11:53:49.412 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 2, 2024-04-25T11:53:49.412 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 2, 2024-04-25T11:53:49.412 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-25T11:53:49.412 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-04-25T11:53:49.412 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-04-25T11:53:49.412 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 4195956, 2024-04-25T11:53:49.413 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 4195815, 2024-04-25T11:53:49.413 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-25T11:53:49.413 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.413 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.413 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.413 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 6 2024-04-25T11:53:49.413 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.413 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-25T11:53:49.414 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 3, 2024-04-25T11:53:49.414 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 3, 2024-04-25T11:53:49.414 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 0, 2024-04-25T11:53:49.414 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 1, 2024-04-25T11:53:49.414 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 1, 2024-04-25T11:53:49.414 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 0, 2024-04-25T11:53:49.414 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 0, 2024-04-25T11:53:49.415 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 0, 2024-04-25T11:53:49.415 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 0, 2024-04-25T11:53:49.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 0, 2024-04-25T11:53:49.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 0, 2024-04-25T11:53:49.415 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 0, 2024-04-25T11:53:49.415 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 50416, 2024-04-25T11:53:49.415 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 215416 2024-04-25T11:53:49.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.416 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-25T11:53:49.416 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 0, 2024-04-25T11:53:49.416 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-04-25T11:53:49.416 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 0, 2024-04-25T11:53:49.416 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 0, 2024-04-25T11:53:49.416 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-04-25T11:53:49.416 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-25T11:53:49.417 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.417 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.417 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.417 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-25T11:53:49.417 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.417 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.418 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.418 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.418 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-25T11:53:49.418 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.418 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.419 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.419 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-25T11:53:49.419 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.419 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.419 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.419 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-25T11:53:49.419 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.420 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.420 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.420 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-25T11:53:49.420 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.421 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.421 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.421 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.421 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-25T11:53:49.422 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.422 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.422 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.423 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-25T11:53:49.423 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.423 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.423 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.423 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-25T11:53:49.424 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.424 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.424 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.424 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-25T11:53:49.424 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.424 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.424 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.425 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-25T11:53:49.425 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.425 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.425 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.425 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.425 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-25T11:53:49.425 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.426 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.426 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.427 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-25T11:53:49.427 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.427 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.427 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.427 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-25T11:53:49.427 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.427 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.427 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.428 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-25T11:53:49.428 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.428 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.428 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.428 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.428 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-25T11:53:49.428 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.428 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.429 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.429 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-25T11:53:49.429 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.429 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.429 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.429 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-25T11:53:49.430 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.430 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.430 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.430 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.430 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-25T11:53:49.430 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.431 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.431 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.431 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-25T11:53:49.431 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.431 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.431 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.431 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-25T11:53:49.432 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.432 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.432 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.432 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.432 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-25T11:53:49.432 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.432 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.432 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.433 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-25T11:53:49.433 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.433 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.433 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.433 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-25T11:53:49.433 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.433 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.434 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.434 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-25T11:53:49.434 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.434 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.434 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.434 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.434 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-25T11:53:49.435 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.435 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.435 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.435 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-25T11:53:49.435 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.435 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.435 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.436 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-25T11:53:49.436 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.436 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.436 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.436 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-25T11:53:49.436 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.436 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.436 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.437 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:53:49.437 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:53:49.437 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-25T11:53:49.437 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.437 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.437 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.437 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:49.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.438 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-25T11:53:49.438 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 0, 2024-04-25T11:53:49.438 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 2, 2024-04-25T11:53:49.438 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 2, 2024-04-25T11:53:49.438 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 2, 2024-04-25T11:53:49.438 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-25T11:53:49.438 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 0, 2024-04-25T11:53:49.438 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 0, 2024-04-25T11:53:49.439 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 1237, 2024-04-25T11:53:49.439 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-25T11:53:49.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.439 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-25T11:53:49.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:53:49.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-25T11:53:49.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:53:49.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:53:49.439 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:53:49.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:53:49.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:53:49.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:53:49.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:53:49.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:53:49.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:53:49.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:53:49.440 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:53:49.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:53:49.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:53:49.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-25T11:53:49.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:53:49.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-25T11:53:49.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:53:49.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:53:49.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:53:49.441 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:53:49.442 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:53:49.442 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:53:49.442 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:53:49.442 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-25T11:53:49.442 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:53:49.442 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:53:49.442 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:53:49.442 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-25T11:53:49.443 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-25T11:53:49.443 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-25T11:53:49.443 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:53:49.443 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:53:49.443 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:53:49.443 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:53:49.443 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 189393, 2024-04-25T11:53:49.443 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 51, 2024-04-25T11:53:49.443 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:53:49.444 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-25T11:53:49.444 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-25T11:53:49.444 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-25T11:53:49.444 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:53:49.444 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-25T11:53:49.444 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:53:49.444 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-25T11:53:49.444 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 8032, 2024-04-25T11:53:49.444 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 167, 2024-04-25T11:53:49.445 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:53:49.445 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:53:49.445 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-25T11:53:49.445 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-25T11:53:49.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 8512, 2024-04-25T11:53:49.445 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 13, 2024-04-25T11:53:49.445 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-25T11:53:49.445 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-25T11:53:49.446 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-25T11:53:49.446 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-25T11:53:49.446 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.446 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-25T11:53:49.446 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-25T11:53:49.446 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-25T11:53:49.446 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 2050, 2024-04-25T11:53:49.446 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 21, 2024-04-25T11:53:49.447 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 3, 2024-04-25T11:53:49.447 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 2050, 2024-04-25T11:53:49.447 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-25T11:53:49.447 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2050, 2024-04-25T11:53:49.447 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.122971521, 2024-04-25T11:53:49.447 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000547790 2024-04-25T11:53:49.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.447 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-25T11:53:49.447 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-25T11:53:49.448 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2050, 2024-04-25T11:53:49.448 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 2051 2024-04-25T11:53:49.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.448 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 2050, 2024-04-25T11:53:49.448 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 2050, 2024-04-25T11:53:49.448 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 0, 2024-04-25T11:53:49.448 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-25T11:53:49.448 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-25T11:53:49.448 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 1325, 2024-04-25T11:53:49.449 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 0, 2024-04-25T11:53:49.449 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 723, 2024-04-25T11:53:49.449 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 0, 2024-04-25T11:53:49.449 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 0, 2024-04-25T11:53:49.449 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-25T11:53:49.449 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-25T11:53:49.449 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-04-25T11:53:49.449 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-04-25T11:53:49.450 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 0, 2024-04-25T11:53:49.450 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-25T11:53:49.450 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-25T11:53:49.450 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-25T11:53:49.450 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 1, 2024-04-25T11:53:49.450 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 0, 2024-04-25T11:53:49.450 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:53:49.450 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-25T11:53:49.450 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:53:49.451 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-25T11:53:49.451 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-25T11:53:49.451 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-25T11:53:49.451 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:53:49.451 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-25T11:53:49.451 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:53:49.451 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-04-25T11:53:49.451 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-25T11:53:49.451 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-25T11:53:49.452 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-25T11:53:49.452 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-25T11:53:49.452 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-25T11:53:49.452 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-25T11:53:49.452 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-25T11:53:49.452 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-25T11:53:49.452 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-25T11:53:49.452 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-25T11:53:49.453 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-25T11:53:49.453 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-25T11:53:49.453 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-25T11:53:49.453 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-25T11:53:49.453 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-25T11:53:49.453 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-25T11:53:49.453 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 97, 2024-04-25T11:53:49.453 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-25T11:53:49.453 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 11, 2024-04-25T11:53:49.454 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 5, 2024-04-25T11:53:49.454 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 5, 2024-04-25T11:53:49.454 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-04-25T11:53:49.454 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-25T11:53:49.454 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 0, 2024-04-25T11:53:49.454 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 2, 2024-04-25T11:53:49.454 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 0 2024-04-25T11:53:49.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.454 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-25T11:53:49.455 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 0, 2024-04-25T11:53:49.455 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:53:49.455 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-25T11:53:49.455 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-25T11:53:49.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.455 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-25T11:53:49.455 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-25T11:53:49.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.456 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-25T11:53:49.456 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-25T11:53:49.456 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:53:49.456 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-25T11:53:49.456 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 0, 2024-04-25T11:53:49.456 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 0, 2024-04-25T11:53:49.456 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 0, 2024-04-25T11:53:49.456 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-25T11:53:49.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.457 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:53:49.457 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T11:53:49.457 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-25T11:53:49.457 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T11:53:49.457 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 2278, 2024-04-25T11:53:49.457 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1981684, 2024-04-25T11:53:49.457 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T11:53:49.457 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 2278, 2024-04-25T11:53:49.457 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-25T11:53:49.458 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-25T11:53:49.458 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 2278, 2024-04-25T11:53:49.458 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1981684, 2024-04-25T11:53:49.458 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T11:53:49.458 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.458 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.458 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.458 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:49.459 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.459 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-25T11:53:49.459 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T11:53:49.459 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-25T11:53:49.459 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T11:53:49.459 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-25T11:53:49.459 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-25T11:53:49.459 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T11:53:49.459 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-25T11:53:49.460 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 2050, 2024-04-25T11:53:49.460 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 1657, 2024-04-25T11:53:49.460 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 2050, 2024-04-25T11:53:49.460 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1657, 2024-04-25T11:53:49.460 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T11:53:49.460 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.460 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.460 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.461 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:49.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.461 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-25T11:53:49.461 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T11:53:49.461 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-25T11:53:49.461 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T11:53:49.461 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-25T11:53:49.461 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-25T11:53:49.461 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T11:53:49.462 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-25T11:53:49.462 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 2050, 2024-04-25T11:53:49.462 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 2050, 2024-04-25T11:53:49.462 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 2050, 2024-04-25T11:53:49.462 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 2050, 2024-04-25T11:53:49.462 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T11:53:49.462 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.462 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.463 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.463 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:49.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.463 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:53:49.463 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T11:53:49.463 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-25T11:53:49.463 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T11:53:49.463 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-25T11:53:49.463 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-25T11:53:49.464 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T11:53:49.464 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-25T11:53:49.464 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-25T11:53:49.464 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-25T11:53:49.464 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-04-25T11:53:49.464 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-04-25T11:53:49.464 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T11:53:49.464 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.464 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.465 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.465 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:49.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:49.465 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x55a9d1fee6e0": { 2024-04-25T11:53:49.465 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T11:53:49.465 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-25T11:53:49.465 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T11:53:49.465 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-25T11:53:49.465 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-25T11:53:49.466 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T11:53:49.466 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-25T11:53:49.466 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-25T11:53:49.466 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-25T11:53:49.466 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-04-25T11:53:49.466 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-04-25T11:53:49.466 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T11:53:49.466 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:49.467 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:49.467 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:49.467 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:49.467 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:49.467 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:53:50.121 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:49 smithi083 ceph-mon[29330]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T11:53:50.149 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-25T11:53:50.149 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-25T11:53:50.150 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-25T11:53:50.150 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-25T11:53:50.150 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T11:53:50.150 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-04-25T11:53:50.150 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T11:53:50.150 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T11:53:50.273 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:49 smithi094 ceph-mon[33032]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T11:53:50.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:49 smithi111 ceph-mon[32784]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T11:53:50.393 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:53:52.040 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:51 smithi083 ceph-mon[29330]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T11:53:52.083 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:53:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:53:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 0, 2024-04-25T11:53:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2, 2024-04-25T11:53:52.083 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 0, 2024-04-25T11:53:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 0, 2024-04-25T11:53:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-04-25T11:53:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-04-25T11:53:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.001185876, 2024-04-25T11:53:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T11:53:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000267078, 2024-04-25T11:53:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:53:52.084 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:53:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:53:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:52.085 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:53:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:53:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:53:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 214, 2024-04-25T11:53:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 203, 2024-04-25T11:53:52.086 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1587472, 2024-04-25T11:53:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 124704, 2024-04-25T11:53:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-04-25T11:53:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-04-25T11:53:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.043477287, 2024-04-25T11:53:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.013760243, 2024-04-25T11:53:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.033761943, 2024-04-25T11:53:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.004367896, 2024-04-25T11:53:52.087 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:53:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 202, 2024-04-25T11:53:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.011473092, 2024-04-25T11:53:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000056797 2024-04-25T11:53:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:53:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:52.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1587472, 2024-04-25T11:53:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 124704 2024-04-25T11:53:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:53:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 6, 2024-04-25T11:53:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 157, 2024-04-25T11:53:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3264, 2024-04-25T11:53:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 183712, 2024-04-25T11:53:52.089 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-04-25T11:53:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-25T11:53:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.014087526, 2024-04-25T11:53:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.009606738, 2024-04-25T11:53:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.002134471, 2024-04-25T11:53:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:53:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:53:52.090 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 155, 2024-04-25T11:53:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.016561737, 2024-04-25T11:53:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000106849 2024-04-25T11:53:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:53:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:52.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T11:53:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 183712 2024-04-25T11:53:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-25T11:53:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-04-25T11:53:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-25T11:53:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-25T11:53:52.092 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:53:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-25T11:53:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:53:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:53:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:53:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:53:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:53:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:53:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:53:52.093 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:53:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:53:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:53:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:53:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:53:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:53:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-25T11:53:52.094 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:53:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-25T11:53:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:53:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:53:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:53:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:53:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:53:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:53:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:53:52.095 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-25T11:53:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:53:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:53:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:53:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-25T11:53:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-25T11:53:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-25T11:53:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:53:52.096 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:53:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:53:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:53:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148977, 2024-04-25T11:53:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-04-25T11:53:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:53:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-25T11:53:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-25T11:53:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-25T11:53:52.097 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:53:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-25T11:53:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:53:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-25T11:53:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-04-25T11:53:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-04-25T11:53:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:53:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:53:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-25T11:53:52.098 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-25T11:53:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-04-25T11:53:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-04-25T11:53:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-25T11:53:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-25T11:53:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-25T11:53:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-25T11:53:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:53:52.099 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T11:53:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-25T11:53:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T11:53:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 216, 2024-04-25T11:53:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1557320, 2024-04-25T11:53:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T11:53:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 216, 2024-04-25T11:53:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-25T11:53:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-25T11:53:52.100 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 216, 2024-04-25T11:53:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1557320, 2024-04-25T11:53:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T11:53:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:52.101 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:52.101 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:53:52.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:51 smithi094 ceph-mon[33032]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T11:53:52.322 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:51 smithi111 ceph-mon[32784]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T11:53:52.727 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T11:53:52.728 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T11:53:52.728 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T11:53:52.728 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T11:53:52.728 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T11:53:52.728 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-25T11:53:52.728 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T11:53:52.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T11:53:52.970 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:53:54.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:53 smithi094 ceph-mon[33032]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T11:53:54.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:53 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:53:54.075 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:53 smithi111 ceph-mon[32784]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T11:53:54.076 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:53 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:53:54.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:53 smithi083 ceph-mon[29330]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T11:53:54.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:53 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T11:53:54.627 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:53:54.627 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:53:54.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 954, 2024-04-25T11:53:54.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 940, 2024-04-25T11:53:54.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1861360, 2024-04-25T11:53:54.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 355596, 2024-04-25T11:53:54.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-04-25T11:53:54.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-04-25T11:53:54.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.127868898, 2024-04-25T11:53:54.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.051018843, 2024-04-25T11:53:54.628 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.049994033, 2024-04-25T11:53:54.629 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.026601158, 2024-04-25T11:53:54.629 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:53:54.629 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 940, 2024-04-25T11:53:54.629 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.053372995, 2024-04-25T11:53:54.629 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000056779 2024-04-25T11:53:54.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.629 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:53:54.629 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.630 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.630 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.630 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1618768, 2024-04-25T11:53:54.630 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 126000 2024-04-25T11:53:54.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.630 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:53:54.630 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1341, 2024-04-25T11:53:54.631 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1491, 2024-04-25T11:53:54.631 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 342982, 2024-04-25T11:53:54.631 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 858243, 2024-04-25T11:53:54.631 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 7, 2024-04-25T11:53:54.631 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-04-25T11:53:54.631 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.097049571, 2024-04-25T11:53:54.631 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.032234817, 2024-04-25T11:53:54.631 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.023713681, 2024-04-25T11:53:54.631 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.033329899, 2024-04-25T11:53:54.632 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:53:54.632 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1488, 2024-04-25T11:53:54.632 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.051565513, 2024-04-25T11:53:54.632 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000034654 2024-04-25T11:53:54.632 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.632 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:53:54.632 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.632 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.632 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.633 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:53:54.633 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 432384 2024-04-25T11:53:54.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.633 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:53:54.633 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 4, 2024-04-25T11:53:54.633 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 6, 2024-04-25T11:53:54.633 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1629, 2024-04-25T11:53:54.634 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 1284, 2024-04-25T11:53:54.634 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-04-25T11:53:54.634 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-04-25T11:53:54.634 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.001347356, 2024-04-25T11:53:54.634 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000062100, 2024-04-25T11:53:54.634 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.001114567, 2024-04-25T11:53:54.634 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000090943, 2024-04-25T11:53:54.634 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:53:54.634 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 4, 2024-04-25T11:53:54.635 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.002180878, 2024-04-25T11:53:54.635 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000545219 2024-04-25T11:53:54.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.635 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:53:54.635 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.635 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.635 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.635 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.636 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:53:54.636 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:53:54.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.636 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-25T11:53:54.636 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-04-25T11:53:54.636 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-25T11:53:54.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.636 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-04-25T11:53:54.636 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-25T11:53:54.637 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-25T11:53:54.637 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3486, 2024-04-25T11:53:54.637 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.086347471, 2024-04-25T11:53:54.637 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000024769 2024-04-25T11:53:54.637 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:54.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.637 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-04-25T11:53:54.637 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-04-25T11:53:54.638 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-04-25T11:53:54.638 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.638 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.638 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.638 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:54.638 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.638 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-04-25T11:53:54.638 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 0, 2024-04-25T11:53:54.638 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 0, 2024-04-25T11:53:54.639 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-04-25T11:53:54.639 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.639 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.639 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.639 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-04-25T11:53:54.639 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-04-25T11:53:54.639 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-04-25T11:53:54.640 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-04-25T11:53:54.640 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 0, 2024-04-25T11:53:54.640 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-04-25T11:53:54.640 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-04-25T11:53:54.640 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 0, 2024-04-25T11:53:54.640 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 0, 2024-04-25T11:53:54.640 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 0, 2024-04-25T11:53:54.640 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-04-25T11:53:54.640 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 0, 2024-04-25T11:53:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-04-25T11:53:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 0, 2024-04-25T11:53:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 0, 2024-04-25T11:53:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 1, 2024-04-25T11:53:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 0, 2024-04-25T11:53:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 0, 2024-04-25T11:53:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-04-25T11:53:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-04-25T11:53:54.641 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:53:54.642 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-04-25T11:53:54.642 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-04-25T11:53:54.642 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-04-25T11:53:54.642 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-04-25T11:53:54.642 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-04-25T11:53:54.642 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-04-25T11:53:54.642 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-04-25T11:53:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-04-25T11:53:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-04-25T11:53:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:53:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-04-25T11:53:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 0, 2024-04-25T11:53:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 0, 2024-04-25T11:53:54.643 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-04-25T11:53:54.644 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:53:54.644 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-04-25T11:53:54.644 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:53:54.644 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:53:54.644 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:53:54.644 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:53:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:53:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-04-25T11:53:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:53:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:53:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:53:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:53:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:53:54.645 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:53:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 0, 2024-04-25T11:53:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:53:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 0, 2024-04-25T11:53:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 0 2024-04-25T11:53:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-04-25T11:53:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-04-25T11:53:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-04-25T11:53:54.646 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-04-25T11:53:54.647 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-04-25T11:53:54.647 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-04-25T11:53:54.647 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-04-25T11:53:54.647 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-04-25T11:53:54.647 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:53:54.647 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 0, 2024-04-25T11:53:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 0, 2024-04-25T11:53:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-04-25T11:53:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-04-25T11:53:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-04-25T11:53:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:53:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:53:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-04-25T11:53:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-04-25T11:53:54.648 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:53:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:53:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:53:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-04-25T11:53:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-04-25T11:53:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:53:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-04-25T11:53:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-04-25T11:53:54.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-04-25T11:53:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-04-25T11:53:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 0, 2024-04-25T11:53:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-04-25T11:53:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-04-25T11:53:54.650 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 6, 2024-04-25T11:53:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-04-25T11:53:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-04-25T11:53:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 0, 2024-04-25T11:53:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-04-25T11:53:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-04-25T11:53:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 2, 2024-04-25T11:53:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 2, 2024-04-25T11:53:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-04-25T11:53:54.651 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-04-25T11:53:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-04-25T11:53:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 4195956, 2024-04-25T11:53:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 4195815, 2024-04-25T11:53:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-04-25T11:53:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.652 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 6 2024-04-25T11:53:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-04-25T11:53:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 3, 2024-04-25T11:53:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 3, 2024-04-25T11:53:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 0, 2024-04-25T11:53:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 1, 2024-04-25T11:53:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 1, 2024-04-25T11:53:54.653 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 0, 2024-04-25T11:53:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 0, 2024-04-25T11:53:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 0, 2024-04-25T11:53:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 0, 2024-04-25T11:53:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 0, 2024-04-25T11:53:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 0, 2024-04-25T11:53:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 0, 2024-04-25T11:53:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 52548, 2024-04-25T11:53:54.654 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 215416 2024-04-25T11:53:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-04-25T11:53:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 0, 2024-04-25T11:53:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-04-25T11:53:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 0, 2024-04-25T11:53:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 0, 2024-04-25T11:53:54.655 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-04-25T11:53:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-04-25T11:53:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-04-25T11:53:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.656 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.657 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-04-25T11:53:54.657 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.657 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.657 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.657 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-04-25T11:53:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-04-25T11:53:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.659 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-04-25T11:53:54.659 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.659 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.659 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.659 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-04-25T11:53:54.659 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.659 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-04-25T11:53:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.660 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-04-25T11:53:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-04-25T11:53:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.661 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.662 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-04-25T11:53:54.662 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.662 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.662 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.662 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-04-25T11:53:54.662 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.663 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-04-25T11:53:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.664 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-04-25T11:53:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-04-25T11:53:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.665 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-04-25T11:53:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-04-25T11:53:54.666 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-04-25T11:53:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-04-25T11:53:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-04-25T11:53:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-04-25T11:53:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-04-25T11:53:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.669 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-04-25T11:53:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.670 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-04-25T11:53:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-04-25T11:53:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.672 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-04-25T11:53:54.672 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.672 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.672 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.672 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-04-25T11:53:54.672 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.672 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.672 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.673 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-04-25T11:53:54.673 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.673 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.673 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.673 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.673 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-04-25T11:53:54.673 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.674 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.674 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.674 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:53:54.674 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:53:54.674 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-04-25T11:53:54.674 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.675 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.675 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.675 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:54.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.675 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-04-25T11:53:54.675 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 0, 2024-04-25T11:53:54.675 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 2, 2024-04-25T11:53:54.675 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 2, 2024-04-25T11:53:54.675 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 2, 2024-04-25T11:53:54.676 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-04-25T11:53:54.676 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 0, 2024-04-25T11:53:54.676 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 0, 2024-04-25T11:53:54.676 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 1239, 2024-04-25T11:53:54.676 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-04-25T11:53:54.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.676 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-25T11:53:54.676 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:53:54.676 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-25T11:53:54.677 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:53:54.677 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:53:54.677 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:53:54.677 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:53:54.677 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:53:54.677 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:53:54.677 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:53:54.677 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:53:54.678 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:53:54.678 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:53:54.678 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:53:54.678 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:53:54.678 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:53:54.678 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-25T11:53:54.678 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:53:54.678 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-25T11:53:54.679 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:53:54.679 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:53:54.679 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:53:54.679 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:53:54.679 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:53:54.679 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:53:54.679 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:53:54.680 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-25T11:53:54.680 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:53:54.680 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:53:54.680 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:53:54.680 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-25T11:53:54.680 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-25T11:53:54.680 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-25T11:53:54.680 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:53:54.680 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:53:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:53:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:53:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 197513, 2024-04-25T11:53:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 56, 2024-04-25T11:53:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:53:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-25T11:53:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-25T11:53:54.681 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-25T11:53:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:53:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-25T11:53:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:53:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-25T11:53:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 8032, 2024-04-25T11:53:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 167, 2024-04-25T11:53:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:53:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:53:54.682 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-25T11:53:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-25T11:53:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 8512, 2024-04-25T11:53:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 13, 2024-04-25T11:53:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-25T11:53:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-25T11:53:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-25T11:53:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-25T11:53:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.683 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-04-25T11:53:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-04-25T11:53:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-04-25T11:53:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 2065, 2024-04-25T11:53:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 21, 2024-04-25T11:53:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 3, 2024-04-25T11:53:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 2065, 2024-04-25T11:53:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-04-25T11:53:54.684 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2065, 2024-04-25T11:53:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.078972633, 2024-04-25T11:53:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000522504 2024-04-25T11:53:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-04-25T11:53:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-04-25T11:53:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2065, 2024-04-25T11:53:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 2066 2024-04-25T11:53:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.685 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 2065, 2024-04-25T11:53:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 2065, 2024-04-25T11:53:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 0, 2024-04-25T11:53:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-04-25T11:53:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-04-25T11:53:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 1335, 2024-04-25T11:53:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 0, 2024-04-25T11:53:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 728, 2024-04-25T11:53:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 0, 2024-04-25T11:53:54.686 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 0, 2024-04-25T11:53:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-04-25T11:53:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-04-25T11:53:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-04-25T11:53:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-04-25T11:53:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 0, 2024-04-25T11:53:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-04-25T11:53:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-04-25T11:53:54.687 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-04-25T11:53:54.688 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 1, 2024-04-25T11:53:54.688 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 0, 2024-04-25T11:53:54.688 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:53:54.690 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-04-25T11:53:54.690 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:53:54.690 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-04-25T11:53:54.690 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-04-25T11:53:54.690 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-04-25T11:53:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:53:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-04-25T11:53:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:53:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-04-25T11:53:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-04-25T11:53:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-04-25T11:53:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-04-25T11:53:54.691 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-04-25T11:53:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-04-25T11:53:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-04-25T11:53:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-04-25T11:53:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-04-25T11:53:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-04-25T11:53:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-04-25T11:53:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-04-25T11:53:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-04-25T11:53:54.692 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-04-25T11:53:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-04-25T11:53:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-04-25T11:53:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-04-25T11:53:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 97, 2024-04-25T11:53:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-04-25T11:53:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 11, 2024-04-25T11:53:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 6, 2024-04-25T11:53:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 6, 2024-04-25T11:53:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 0, 2024-04-25T11:53:54.693 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-04-25T11:53:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 0, 2024-04-25T11:53:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 2, 2024-04-25T11:53:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 0 2024-04-25T11:53:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-04-25T11:53:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 0, 2024-04-25T11:53:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:53:54.694 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-04-25T11:53:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-04-25T11:53:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-04-25T11:53:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-04-25T11:53:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-04-25T11:53:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-04-25T11:53:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:53:54.695 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-04-25T11:53:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 0, 2024-04-25T11:53:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 0, 2024-04-25T11:53:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 0, 2024-04-25T11:53:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-04-25T11:53:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:53:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T11:53:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-25T11:53:54.696 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T11:53:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 2295, 2024-04-25T11:53:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1984814, 2024-04-25T11:53:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T11:53:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 2295, 2024-04-25T11:53:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-25T11:53:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-25T11:53:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 2295, 2024-04-25T11:53:54.697 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1984814, 2024-04-25T11:53:54.698 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T11:53:54.698 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.698 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.698 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.698 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-04-25T11:53:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T11:53:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-25T11:53:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T11:53:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-25T11:53:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-25T11:53:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T11:53:54.699 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-25T11:53:54.700 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 2065, 2024-04-25T11:53:54.700 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 1657, 2024-04-25T11:53:54.700 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 2065, 2024-04-25T11:53:54.700 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1657, 2024-04-25T11:53:54.700 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T11:53:54.700 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-04-25T11:53:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T11:53:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-04-25T11:53:54.701 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T11:53:54.702 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-25T11:53:54.702 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-25T11:53:54.702 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T11:53:54.702 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-25T11:53:54.702 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 2065, 2024-04-25T11:53:54.702 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 2065, 2024-04-25T11:53:54.702 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 2065, 2024-04-25T11:53:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 2065, 2024-04-25T11:53:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T11:53:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:53:54.703 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T11:53:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-25T11:53:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T11:53:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-25T11:53:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-25T11:53:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T11:53:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-25T11:53:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-25T11:53:54.704 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-25T11:53:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-04-25T11:53:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-04-25T11:53:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T11:53:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:54.705 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x55d8cd4e26e0": { 2024-04-25T11:53:54.706 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T11:53:54.706 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-04-25T11:53:54.706 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T11:53:54.706 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-04-25T11:53:54.706 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-04-25T11:53:54.706 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T11:53:54.706 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-04-25T11:53:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-25T11:53:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-25T11:53:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-04-25T11:53:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-04-25T11:53:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T11:53:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:54.707 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:54.708 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:54.708 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:53:55.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T11:53:55.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:55.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:55.657 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:55.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:55.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-25T11:53:55.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: Unable to set osd_memory_target on smithi094 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T11:53:55.658 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi111 to 176.2M 2024-04-25T11:53:55.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:55 smithi083 ceph-mon[29330]: Unable to set osd_memory_target on smithi111 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-25T11:53:55.731 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-25T11:53:55.732 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-25T11:53:55.732 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-25T11:53:55.732 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-25T11:53:55.732 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T11:53:55.732 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-04-25T11:53:55.732 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T11:53:55.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T11:53:55.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T11:53:55.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:55.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:55.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:55.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:55.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.824 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.824 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.824 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-25T11:53:55.824 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: Unable to set osd_memory_target on smithi094 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T11:53:55.824 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.824 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.824 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.824 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:55.824 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi111 to 176.2M 2024-04-25T11:53:55.825 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:55 smithi111 ceph-mon[32784]: Unable to set osd_memory_target on smithi111 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-25T11:53:56.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T11:53:56.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:56.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:56.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:56.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:56.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:56.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:56.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:56.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:56.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-04-25T11:53:56.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: Unable to set osd_memory_target on smithi094 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T11:53:56.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:56.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:56.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:56.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:56.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi111 to 176.2M 2024-04-25T11:53:56.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:55 smithi094 ceph-mon[33032]: Unable to set osd_memory_target on smithi111 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-25T11:53:56.080 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T11:53:57.729 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-25T11:53:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:53:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 0, 2024-04-25T11:53:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2, 2024-04-25T11:53:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 0, 2024-04-25T11:53:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 0, 2024-04-25T11:53:57.729 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-04-25T11:53:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-04-25T11:53:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.001134663, 2024-04-25T11:53:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T11:53:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000199675, 2024-04-25T11:53:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:53:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:53:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:57.730 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:53:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:53:57.731 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:53:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:53:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2, 2024-04-25T11:53:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 157, 2024-04-25T11:53:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 320, 2024-04-25T11:53:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 183616, 2024-04-25T11:53:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-04-25T11:53:57.732 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-25T11:53:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.013801417, 2024-04-25T11:53:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.009727305, 2024-04-25T11:53:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.001885586, 2024-04-25T11:53:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T11:53:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:53:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 154, 2024-04-25T11:53:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.011762199, 2024-04-25T11:53:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000076377 2024-04-25T11:53:57.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:53:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:53:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 183616 2024-04-25T11:53:57.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:53:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 220, 2024-04-25T11:53:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 207, 2024-04-25T11:53:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1590960, 2024-04-25T11:53:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 126432, 2024-04-25T11:53:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-04-25T11:53:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-04-25T11:53:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.043678523, 2024-04-25T11:53:57.735 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.014478828, 2024-04-25T11:53:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.033039606, 2024-04-25T11:53:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.004443565, 2024-04-25T11:53:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:53:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 207, 2024-04-25T11:53:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.011457635, 2024-04-25T11:53:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000055350 2024-04-25T11:53:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:53:57.736 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1590960, 2024-04-25T11:53:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 126432 2024-04-25T11:53:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-25T11:53:57.737 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-04-25T11:53:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-25T11:53:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-25T11:53:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:53:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-25T11:53:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:53:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:53:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:53:57.738 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:53:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:53:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:53:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:53:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:53:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:53:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:53:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:53:57.739 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:53:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:53:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-25T11:53:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:53:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-25T11:53:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:53:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:53:57.740 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:53:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:53:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:53:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:53:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:53:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-25T11:53:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:53:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:53:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:53:57.741 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-25T11:53:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-25T11:53:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-25T11:53:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:53:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:53:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:53:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:53:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148977, 2024-04-25T11:53:57.742 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-04-25T11:53:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:53:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-25T11:53:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-25T11:53:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-25T11:53:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:53:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-25T11:53:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:53:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-25T11:53:57.743 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-04-25T11:53:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-04-25T11:53:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:53:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:53:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-25T11:53:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-25T11:53:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-04-25T11:53:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-04-25T11:53:57.744 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-25T11:53:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-25T11:53:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-25T11:53:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-25T11:53:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-25T11:53:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:53:57.745 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-25T11:53:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-25T11:53:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-25T11:53:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 218, 2024-04-25T11:53:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1557580, 2024-04-25T11:53:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-25T11:53:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 218, 2024-04-25T11:53:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-25T11:53:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-25T11:53:57.746 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 218, 2024-04-25T11:53:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1557580, 2024-04-25T11:53:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-25T11:53:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-25T11:53:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-25T11:53:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-25T11:53:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:57.747 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-25T11:53:57.747 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-25T11:53:58.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:57 smithi094 ceph-mon[33032]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T11:53:58.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:57 smithi111 ceph-mon[32784]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T11:53:58.156 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:57 smithi083 ceph-mon[29330]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T11:53:58.428 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T11:53:58.429 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T11:53:58.429 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T11:53:58.429 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T11:53:58.429 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T11:53:58.429 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-25T11:53:58.429 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T11:53:58.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T11:53:58.683 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:53:59.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:59 smithi083 ceph-mon[29330]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:59.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:59 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:59.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:59 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:59.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:59 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:59.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:59 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:59.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:59 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:59.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:59 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:59.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:59 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:53:59.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:59 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:53:59.908 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:53:59 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:59.913 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:59 smithi094 ceph-mon[33032]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:53:59.913 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:59 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:59.913 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:59 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:53:59.913 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:59 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:59.913 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:59 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:59.914 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:59 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:59.914 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:59 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:53:59.914 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:59 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:53:59.914 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:59 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:53:59.914 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:53:59 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:54:00.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:59 smithi111 ceph-mon[32784]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:54:00.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:59 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:54:00.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:59 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:54:00.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:59 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:54:00.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:59 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:54:00.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:59 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:54:00.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:59 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T11:54:00.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:59 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T11:54:00.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:59 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T11:54:00.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:53:59 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' 2024-04-25T11:54:00.465 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T11:54:00.466 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:54:00.466 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1210, 2024-04-25T11:54:00.466 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 937, 2024-04-25T11:54:00.466 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 400831, 2024-04-25T11:54:00.466 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 655847, 2024-04-25T11:54:00.466 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 11, 2024-04-25T11:54:00.466 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 5, 2024-04-25T11:54:00.466 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.132370776, 2024-04-25T11:54:00.467 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.046050819, 2024-04-25T11:54:00.467 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.042448369, 2024-04-25T11:54:00.467 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.035204384, 2024-04-25T11:54:00.467 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:00.467 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 939, 2024-04-25T11:54:00.467 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.063969922, 2024-04-25T11:54:00.467 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000068125 2024-04-25T11:54:00.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.468 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:00.468 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1470, 2024-04-25T11:54:00.468 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000301167, 2024-04-25T11:54:00.468 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000204 2024-04-25T11:54:00.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.468 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T11:54:00.468 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 311920 2024-04-25T11:54:00.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.468 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:54:00.469 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 2237, 2024-04-25T11:54:00.469 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 740, 2024-04-25T11:54:00.469 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 831706, 2024-04-25T11:54:00.469 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 420804, 2024-04-25T11:54:00.469 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 15, 2024-04-25T11:54:00.469 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 8, 2024-04-25T11:54:00.469 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.267094907, 2024-04-25T11:54:00.469 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.061841981, 2024-04-25T11:54:00.469 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.093871596, 2024-04-25T11:54:00.470 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.105572039, 2024-04-25T11:54:00.470 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:00.470 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 740, 2024-04-25T11:54:00.470 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.116457907, 2024-04-25T11:54:00.470 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000157375 2024-04-25T11:54:00.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.470 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:00.470 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2540, 2024-04-25T11:54:00.471 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000520701, 2024-04-25T11:54:00.471 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000205 2024-04-25T11:54:00.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.471 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 414624, 2024-04-25T11:54:00.471 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 245088 2024-04-25T11:54:00.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.471 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:54:00.471 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1025, 2024-04-25T11:54:00.471 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 723, 2024-04-25T11:54:00.472 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1848359, 2024-04-25T11:54:00.472 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 298652, 2024-04-25T11:54:00.472 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 12, 2024-04-25T11:54:00.472 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 9, 2024-04-25T11:54:00.472 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.126401884, 2024-04-25T11:54:00.472 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.037094133, 2024-04-25T11:54:00.472 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.090781366, 2024-04-25T11:54:00.472 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.029129594, 2024-04-25T11:54:00.472 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:00.473 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 716, 2024-04-25T11:54:00.473 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.116510118, 2024-04-25T11:54:00.473 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000162723 2024-04-25T11:54:00.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.473 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:00.473 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 844, 2024-04-25T11:54:00.473 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000235766, 2024-04-25T11:54:00.473 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000279 2024-04-25T11:54:00.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.474 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1691472, 2024-04-25T11:54:00.474 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 134240 2024-04-25T11:54:00.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.474 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-25T11:54:00.474 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-04-25T11:54:00.474 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-25T11:54:00.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.474 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-04-25T11:54:00.474 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-25T11:54:00.475 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-25T11:54:00.475 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 220, 2024-04-25T11:54:00.475 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.043183795, 2024-04-25T11:54:00.475 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000196289 2024-04-25T11:54:00.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:00.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.475 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-04-25T11:54:00.475 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-25T11:54:00.476 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-25T11:54:00.476 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-04-25T11:54:00.476 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000010623, 2024-04-25T11:54:00.476 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000005311 2024-04-25T11:54:00.476 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:00.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.476 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-25T11:54:00.476 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 146, 2024-04-25T11:54:00.476 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 146, 2024-04-25T11:54:00.477 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-04-25T11:54:00.477 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 146, 2024-04-25T11:54:00.477 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.121978099, 2024-04-25T11:54:00.477 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000835466 2024-04-25T11:54:00.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.477 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-04-25T11:54:00.477 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 0, 2024-04-25T11:54:00.477 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-04-25T11:54:00.477 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-04-25T11:54:00.478 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 12, 2024-04-25T11:54:00.478 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-04-25T11:54:00.478 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-04-25T11:54:00.478 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 25, 2024-04-25T11:54:00.478 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 1, 2024-04-25T11:54:00.478 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 24, 2024-04-25T11:54:00.478 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-04-25T11:54:00.478 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 22, 2024-04-25T11:54:00.479 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 0, 2024-04-25T11:54:00.479 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 2, 2024-04-25T11:54:00.479 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 2, 2024-04-25T11:54:00.479 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 2, 2024-04-25T11:54:00.479 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 183, 2024-04-25T11:54:00.479 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 177, 2024-04-25T11:54:00.479 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 0, 2024-04-25T11:54:00.479 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 0, 2024-04-25T11:54:00.479 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:54:00.480 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-04-25T11:54:00.480 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-04-25T11:54:00.480 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-04-25T11:54:00.480 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-04-25T11:54:00.480 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-04-25T11:54:00.480 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-04-25T11:54:00.480 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-04-25T11:54:00.480 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-04-25T11:54:00.480 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-04-25T11:54:00.481 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:54:00.481 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-04-25T11:54:00.481 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 4, 2024-04-25T11:54:00.481 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 234, 2024-04-25T11:54:00.481 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 6, 2024-04-25T11:54:00.481 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:54:00.481 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-04-25T11:54:00.481 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:54:00.482 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:54:00.482 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:54:00.482 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:54:00.482 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:54:00.482 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-04-25T11:54:00.482 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:54:00.482 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 8, 2024-04-25T11:54:00.482 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 5, 2024-04-25T11:54:00.482 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:54:00.483 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:54:00.483 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:54:00.483 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 14, 2024-04-25T11:54:00.483 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 2, 2024-04-25T11:54:00.483 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 0, 2024-04-25T11:54:00.483 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 17 2024-04-25T11:54:00.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.483 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-04-25T11:54:00.483 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-04-25T11:54:00.484 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 0, 2024-04-25T11:54:00.484 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 0, 2024-04-25T11:54:00.484 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 0, 2024-04-25T11:54:00.484 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 7, 2024-04-25T11:54:00.484 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-04-25T11:54:00.484 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-04-25T11:54:00.484 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:54:00.484 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-04-25T11:54:00.485 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-04-25T11:54:00.485 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-04-25T11:54:00.485 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-04-25T11:54:00.485 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-04-25T11:54:00.485 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:54:00.485 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:54:00.485 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-04-25T11:54:00.485 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-04-25T11:54:00.485 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:54:00.486 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:54:00.486 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:54:00.486 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-04-25T11:54:00.486 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-04-25T11:54:00.486 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:54:00.486 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-04-25T11:54:00.486 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-04-25T11:54:00.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.487 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-04-25T11:54:00.487 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-04-25T11:54:00.487 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 98, 2024-04-25T11:54:00.487 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-04-25T11:54:00.487 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-04-25T11:54:00.487 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 98, 2024-04-25T11:54:00.487 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-04-25T11:54:00.487 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-04-25T11:54:00.487 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 2, 2024-04-25T11:54:00.488 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-04-25T11:54:00.488 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-04-25T11:54:00.488 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 2, 2024-04-25T11:54:00.488 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 2, 2024-04-25T11:54:00.488 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-04-25T11:54:00.488 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-04-25T11:54:00.488 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-04-25T11:54:00.488 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 4281800, 2024-04-25T11:54:00.488 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 0, 2024-04-25T11:54:00.489 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-04-25T11:54:00.489 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 79, 2024-04-25T11:54:00.489 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.137272021, 2024-04-25T11:54:00.489 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.001737620 2024-04-25T11:54:00.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.489 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 0 2024-04-25T11:54:00.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.489 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-04-25T11:54:00.489 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 23, 2024-04-25T11:54:00.490 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 23, 2024-04-25T11:54:00.490 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 0, 2024-04-25T11:54:00.490 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 18, 2024-04-25T11:54:00.490 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 18, 2024-04-25T11:54:00.490 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 0, 2024-04-25T11:54:00.490 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 25, 2024-04-25T11:54:00.490 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 25, 2024-04-25T11:54:00.490 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 0, 2024-04-25T11:54:00.490 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 2, 2024-04-25T11:54:00.491 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 13, 2024-04-25T11:54:00.491 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 11, 2024-04-25T11:54:00.491 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 54272, 2024-04-25T11:54:00.491 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 231800 2024-04-25T11:54:00.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.491 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-04-25T11:54:00.491 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 184, 2024-04-25T11:54:00.491 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 1, 2024-04-25T11:54:00.491 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 146, 2024-04-25T11:54:00.492 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 31, 2024-04-25T11:54:00.492 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 2, 2024-04-25T11:54:00.492 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-04-25T11:54:00.492 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 4, 2024-04-25T11:54:00.492 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.003429589, 2024-04-25T11:54:00.492 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000857397 2024-04-25T11:54:00.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.492 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-04-25T11:54:00.492 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 101, 2024-04-25T11:54:00.493 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.018867284, 2024-04-25T11:54:00.493 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000186804 2024-04-25T11:54:00.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.493 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-04-25T11:54:00.493 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.493 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.493 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.493 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-04-25T11:54:00.494 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.494 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.494 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.494 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-04-25T11:54:00.494 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.494 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.494 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.495 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-04-25T11:54:00.495 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.495 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.495 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.495 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-04-25T11:54:00.495 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.495 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.496 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.496 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-04-25T11:54:00.496 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.496 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.496 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.496 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-04-25T11:54:00.497 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.497 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.497 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.497 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-04-25T11:54:00.497 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.497 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.497 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.498 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-04-25T11:54:00.498 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 10, 2024-04-25T11:54:00.498 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.001375976, 2024-04-25T11:54:00.498 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000137597 2024-04-25T11:54:00.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.499 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-04-25T11:54:00.499 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 6, 2024-04-25T11:54:00.499 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.007799658, 2024-04-25T11:54:00.500 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.001299943 2024-04-25T11:54:00.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.500 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-04-25T11:54:00.500 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.500 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.500 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.500 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-04-25T11:54:00.501 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 10, 2024-04-25T11:54:00.501 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.050020301, 2024-04-25T11:54:00.501 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.005002030 2024-04-25T11:54:00.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.501 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-04-25T11:54:00.501 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1, 2024-04-25T11:54:00.501 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000200215, 2024-04-25T11:54:00.502 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000200215 2024-04-25T11:54:00.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.502 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-04-25T11:54:00.502 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.502 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.502 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.502 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-04-25T11:54:00.503 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-04-25T11:54:00.503 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.002784323, 2024-04-25T11:54:00.503 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.001392161 2024-04-25T11:54:00.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.503 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-04-25T11:54:00.503 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.503 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.503 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.504 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-04-25T11:54:00.504 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.504 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.504 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.504 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-04-25T11:54:00.504 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.504 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.504 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.505 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-04-25T11:54:00.505 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 4, 2024-04-25T11:54:00.505 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.017635885, 2024-04-25T11:54:00.505 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.004408971 2024-04-25T11:54:00.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.505 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-04-25T11:54:00.506 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.506 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.506 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.506 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-04-25T11:54:00.506 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1, 2024-04-25T11:54:00.506 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000809892, 2024-04-25T11:54:00.506 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000809892 2024-04-25T11:54:00.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.507 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-04-25T11:54:00.507 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.507 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.507 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.507 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-04-25T11:54:00.507 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.508 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.508 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.508 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-04-25T11:54:00.508 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.508 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.508 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.508 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-04-25T11:54:00.509 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 7, 2024-04-25T11:54:00.509 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.019054976, 2024-04-25T11:54:00.509 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.002722139 2024-04-25T11:54:00.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.509 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-04-25T11:54:00.509 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.509 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.509 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.510 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-04-25T11:54:00.510 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.510 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.510 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.510 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:54:00.510 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:54:00.510 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-04-25T11:54:00.511 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.511 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.511 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.511 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:00.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.511 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-04-25T11:54:00.511 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 2, 2024-04-25T11:54:00.511 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 5, 2024-04-25T11:54:00.511 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 3, 2024-04-25T11:54:00.512 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 2, 2024-04-25T11:54:00.512 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-04-25T11:54:00.512 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 27, 2024-04-25T11:54:00.512 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 13, 2024-04-25T11:54:00.512 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 610, 2024-04-25T11:54:00.512 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-04-25T11:54:00.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.512 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-25T11:54:00.513 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:54:00.513 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-25T11:54:00.513 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:54:00.513 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:54:00.513 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:54:00.513 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:54:00.513 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:54:00.513 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:54:00.513 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:54:00.514 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:54:00.514 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:54:00.514 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:54:00.514 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:54:00.514 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:54:00.514 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:54:00.514 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-25T11:54:00.514 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:54:00.515 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-25T11:54:00.515 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:54:00.515 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:54:00.515 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:54:00.515 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:54:00.515 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:54:00.515 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:54:00.515 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:54:00.515 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-25T11:54:00.516 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:54:00.516 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:54:00.516 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:54:00.516 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-25T11:54:00.516 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-25T11:54:00.516 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-25T11:54:00.516 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:54:00.516 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:54:00.516 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:54:00.517 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:54:00.517 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 367227, 2024-04-25T11:54:00.517 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 152, 2024-04-25T11:54:00.517 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 880, 2024-04-25T11:54:00.517 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 10, 2024-04-25T11:54:00.517 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-25T11:54:00.517 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-25T11:54:00.517 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:54:00.517 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-25T11:54:00.518 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:54:00.518 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-25T11:54:00.518 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 8032, 2024-04-25T11:54:00.518 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 167, 2024-04-25T11:54:00.518 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:54:00.518 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:54:00.518 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-25T11:54:00.518 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-25T11:54:00.518 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 94290, 2024-04-25T11:54:00.519 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 556, 2024-04-25T11:54:00.519 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-25T11:54:00.519 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-25T11:54:00.519 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-25T11:54:00.519 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-25T11:54:00.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.519 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-04-25T11:54:00.519 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-04-25T11:54:00.519 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-04-25T11:54:00.520 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 636, 2024-04-25T11:54:00.520 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 101359, 2024-04-25T11:54:00.520 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 37, 2024-04-25T11:54:00.520 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 636, 2024-04-25T11:54:00.520 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-04-25T11:54:00.520 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 636, 2024-04-25T11:54:00.520 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 14.792767218, 2024-04-25T11:54:00.520 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.023259067 2024-04-25T11:54:00.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.521 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-04-25T11:54:00.521 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-04-25T11:54:00.521 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 636, 2024-04-25T11:54:00.521 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 667 2024-04-25T11:54:00.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.521 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 636, 2024-04-25T11:54:00.521 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 0, 2024-04-25T11:54:00.521 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 636, 2024-04-25T11:54:00.522 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-04-25T11:54:00.522 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-04-25T11:54:00.522 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 0, 2024-04-25T11:54:00.522 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 11, 2024-04-25T11:54:00.522 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 0, 2024-04-25T11:54:00.522 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 79, 2024-04-25T11:54:00.522 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 19, 2024-04-25T11:54:00.522 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-04-25T11:54:00.522 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-04-25T11:54:00.523 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-04-25T11:54:00.523 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-04-25T11:54:00.523 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 506, 2024-04-25T11:54:00.523 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-04-25T11:54:00.523 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-04-25T11:54:00.523 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-04-25T11:54:00.523 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 0, 2024-04-25T11:54:00.523 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 22, 2024-04-25T11:54:00.523 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:54:00.524 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-04-25T11:54:00.524 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:54:00.524 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-04-25T11:54:00.524 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-04-25T11:54:00.524 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-04-25T11:54:00.524 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:54:00.524 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-04-25T11:54:00.524 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:54:00.524 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-04-25T11:54:00.525 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-04-25T11:54:00.525 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-04-25T11:54:00.525 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-04-25T11:54:00.525 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-04-25T11:54:00.525 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-04-25T11:54:00.525 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-04-25T11:54:00.525 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-04-25T11:54:00.525 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-04-25T11:54:00.526 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-04-25T11:54:00.526 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-04-25T11:54:00.526 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-04-25T11:54:00.526 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-04-25T11:54:00.526 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-04-25T11:54:00.526 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-04-25T11:54:00.526 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-04-25T11:54:00.526 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-04-25T11:54:00.526 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 97, 2024-04-25T11:54:00.527 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-04-25T11:54:00.527 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 12, 2024-04-25T11:54:00.527 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 12, 2024-04-25T11:54:00.527 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 12, 2024-04-25T11:54:00.527 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 0, 2024-04-25T11:54:00.527 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-04-25T11:54:00.527 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 26, 2024-04-25T11:54:00.527 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 0, 2024-04-25T11:54:00.527 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 4 2024-04-25T11:54:00.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.528 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-04-25T11:54:00.528 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 1, 2024-04-25T11:54:00.528 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:54:00.528 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-04-25T11:54:00.528 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-04-25T11:54:00.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.528 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-04-25T11:54:00.528 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-04-25T11:54:00.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.529 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-04-25T11:54:00.529 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-04-25T11:54:00.529 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:54:00.529 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-04-25T11:54:00.529 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-04-25T11:54:00.529 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-04-25T11:54:00.529 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-04-25T11:54:00.529 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-04-25T11:54:00.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.530 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:54:00.530 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:00.530 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T11:54:00.530 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:00.530 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 4468, 2024-04-25T11:54:00.530 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 2614160, 2024-04-25T11:54:00.530 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:00.530 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 4468, 2024-04-25T11:54:00.531 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T11:54:00.531 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T11:54:00.531 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 4468, 2024-04-25T11:54:00.531 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 2614160, 2024-04-25T11:54:00.531 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:00.531 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.531 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.532 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.532 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:00.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.532 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-04-25T11:54:00.532 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:00.532 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T11:54:00.532 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:00.532 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T11:54:00.532 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T11:54:00.533 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:00.533 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:00.533 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 636, 2024-04-25T11:54:00.533 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 101359, 2024-04-25T11:54:00.533 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 636, 2024-04-25T11:54:00.533 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 101359, 2024-04-25T11:54:00.533 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:00.533 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.534 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.534 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:00.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.534 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-04-25T11:54:00.534 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:00.534 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-04-25T11:54:00.534 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:00.534 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T11:54:00.535 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T11:54:00.535 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:00.535 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:00.535 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 636, 2024-04-25T11:54:00.535 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 636, 2024-04-25T11:54:00.535 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 636, 2024-04-25T11:54:00.535 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 636, 2024-04-25T11:54:00.535 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:00.535 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.536 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.536 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.536 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:00.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.536 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:54:00.536 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:00.536 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-25T11:54:00.536 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:00.537 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T11:54:00.537 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T11:54:00.537 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:00.537 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:00.537 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T11:54:00.537 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T11:54:00.537 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-25T11:54:00.537 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-25T11:54:00.537 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:00.538 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.538 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.538 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:00.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:00.538 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x557ce71d8500": { 2024-04-25T11:54:00.538 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:00.538 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-25T11:54:00.539 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:00.539 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 98, 2024-04-25T11:54:00.539 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 87496, 2024-04-25T11:54:00.539 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:00.539 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 98, 2024-04-25T11:54:00.539 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T11:54:00.539 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T11:54:00.539 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 79, 2024-04-25T11:54:00.539 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 87496, 2024-04-25T11:54:00.540 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:00.540 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:00.540 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:00.540 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:00.540 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:00.540 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:00.540 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T11:54:00.773 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:00 smithi094 ceph-mon[33032]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-25T11:54:00.774 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:00 smithi094 ceph-mon[33032]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T11:54:00.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:00 smithi083 ceph-mon[29330]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-25T11:54:00.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:00 smithi083 ceph-mon[29330]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T11:54:01.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:00 smithi111 ceph-mon[32784]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-04-25T11:54:01.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:00 smithi111 ceph-mon[32784]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T11:54:01.243 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-25T11:54:01.243 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T11:54:01.243 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T11:54:01.243 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-25T11:54:01.244 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-25T11:54:01.244 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-04-25T11:54:01.244 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T11:54:01.244 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T11:54:01.483 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:54:01.774 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:01 smithi094 ceph-mon[33032]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:54:01.906 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:01 smithi083 ceph-mon[29330]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:54:02.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:01 smithi111 ceph-mon[32784]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T11:54:03.246 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T11:54:03.246 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:54:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1509, 2024-04-25T11:54:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1509, 2024-04-25T11:54:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 541891, 2024-04-25T11:54:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 479048, 2024-04-25T11:54:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-04-25T11:54:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 2, 2024-04-25T11:54:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.134804477, 2024-04-25T11:54:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.054268628, 2024-04-25T11:54:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.029876763, 2024-04-25T11:54:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.037463268, 2024-04-25T11:54:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1508, 2024-04-25T11:54:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.056653359, 2024-04-25T11:54:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000037568 2024-04-25T11:54:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T11:54:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T11:54:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:54:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 238, 2024-04-25T11:54:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 226, 2024-04-25T11:54:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1619000, 2024-04-25T11:54:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 130250, 2024-04-25T11:54:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-04-25T11:54:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-04-25T11:54:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.045957092, 2024-04-25T11:54:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.014091429, 2024-04-25T11:54:03.251 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.035814239, 2024-04-25T11:54:03.251 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.005350128, 2024-04-25T11:54:03.251 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:03.251 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 225, 2024-04-25T11:54:03.251 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.014349680, 2024-04-25T11:54:03.251 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000063776 2024-04-25T11:54:03.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.251 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1617344, 2024-04-25T11:54:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 128912 2024-04-25T11:54:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:54:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 645, 2024-04-25T11:54:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 800, 2024-04-25T11:54:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 158569, 2024-04-25T11:54:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 640971, 2024-04-25T11:54:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-04-25T11:54:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 4, 2024-04-25T11:54:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.064304477, 2024-04-25T11:54:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.021276387, 2024-04-25T11:54:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.013966253, 2024-04-25T11:54:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.021634479, 2024-04-25T11:54:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 797, 2024-04-25T11:54:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.031238689, 2024-04-25T11:54:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000039195 2024-04-25T11:54:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:54:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 436496 2024-04-25T11:54:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-25T11:54:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-04-25T11:54:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-25T11:54:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-04-25T11:54:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-25T11:54:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-25T11:54:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3651, 2024-04-25T11:54:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.090878097, 2024-04-25T11:54:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000024891 2024-04-25T11:54:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-04-25T11:54:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-25T11:54:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-25T11:54:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-25T11:54:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 0, 2024-04-25T11:54:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 0, 2024-04-25T11:54:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-04-25T11:54:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-04-25T11:54:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 0, 2024-04-25T11:54:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-04-25T11:54:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-04-25T11:54:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 0, 2024-04-25T11:54:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-04-25T11:54:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-04-25T11:54:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 12, 2024-04-25T11:54:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 0, 2024-04-25T11:54:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 12, 2024-04-25T11:54:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-04-25T11:54:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 12, 2024-04-25T11:54:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 0, 2024-04-25T11:54:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 0, 2024-04-25T11:54:03.261 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 0, 2024-04-25T11:54:03.261 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 2, 2024-04-25T11:54:03.261 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 0, 2024-04-25T11:54:03.261 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 0, 2024-04-25T11:54:03.261 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 0, 2024-04-25T11:54:03.261 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 0, 2024-04-25T11:54:03.261 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:54:03.261 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-04-25T11:54:03.261 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-04-25T11:54:03.262 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-04-25T11:54:03.262 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-04-25T11:54:03.262 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-04-25T11:54:03.262 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-04-25T11:54:03.262 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-04-25T11:54:03.262 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-04-25T11:54:03.262 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-04-25T11:54:03.262 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:54:03.262 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-04-25T11:54:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 4, 2024-04-25T11:54:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 234, 2024-04-25T11:54:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 6, 2024-04-25T11:54:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:54:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-04-25T11:54:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:54:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:54:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:54:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:54:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:54:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-04-25T11:54:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:54:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:54:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:54:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:54:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:54:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:54:03.265 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 0, 2024-04-25T11:54:03.265 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:54:03.265 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 0, 2024-04-25T11:54:03.265 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 0 2024-04-25T11:54:03.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.265 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-04-25T11:54:03.265 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-04-25T11:54:03.265 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 0, 2024-04-25T11:54:03.265 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 0, 2024-04-25T11:54:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 0, 2024-04-25T11:54:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-04-25T11:54:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-04-25T11:54:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-04-25T11:54:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:54:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-04-25T11:54:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-04-25T11:54:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-04-25T11:54:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-04-25T11:54:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-04-25T11:54:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:54:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:54:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-04-25T11:54:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-04-25T11:54:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:54:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:54:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:54:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-04-25T11:54:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-04-25T11:54:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:54:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-04-25T11:54:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-04-25T11:54:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-04-25T11:54:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-04-25T11:54:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 0, 2024-04-25T11:54:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-04-25T11:54:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-04-25T11:54:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 104, 2024-04-25T11:54:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-04-25T11:54:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-04-25T11:54:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 0, 2024-04-25T11:54:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-04-25T11:54:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-04-25T11:54:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 2, 2024-04-25T11:54:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 2, 2024-04-25T11:54:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-04-25T11:54:03.271 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-04-25T11:54:03.271 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-04-25T11:54:03.271 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 4284914, 2024-04-25T11:54:03.271 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 4283593, 2024-04-25T11:54:03.271 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-04-25T11:54:03.271 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.271 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.271 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 104 2024-04-25T11:54:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-04-25T11:54:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 13, 2024-04-25T11:54:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 13, 2024-04-25T11:54:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 0, 2024-04-25T11:54:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 8, 2024-04-25T11:54:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 8, 2024-04-25T11:54:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 0, 2024-04-25T11:54:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 12, 2024-04-25T11:54:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 12, 2024-04-25T11:54:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 0, 2024-04-25T11:54:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 0, 2024-04-25T11:54:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 0, 2024-04-25T11:54:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 0, 2024-04-25T11:54:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 55372, 2024-04-25T11:54:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 215416 2024-04-25T11:54:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-04-25T11:54:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 0, 2024-04-25T11:54:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 0, 2024-04-25T11:54:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 0, 2024-04-25T11:54:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 0, 2024-04-25T11:54:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 0, 2024-04-25T11:54:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-04-25T11:54:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-04-25T11:54:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-04-25T11:54:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-04-25T11:54:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-04-25T11:54:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-04-25T11:54:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-04-25T11:54:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-04-25T11:54:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-04-25T11:54:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-04-25T11:54:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-04-25T11:54:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-04-25T11:54:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-04-25T11:54:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-04-25T11:54:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-04-25T11:54:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.284 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-04-25T11:54:03.284 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.284 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.284 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.284 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-04-25T11:54:03.284 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-04-25T11:54:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-04-25T11:54:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-04-25T11:54:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-04-25T11:54:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-04-25T11:54:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-04-25T11:54:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-04-25T11:54:03.289 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.289 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.289 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.289 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-04-25T11:54:03.289 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.289 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.289 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-04-25T11:54:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-04-25T11:54:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-04-25T11:54:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-04-25T11:54:03.293 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.293 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.293 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.293 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:54:03.293 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:54:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-04-25T11:54:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-04-25T11:54:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 2, 2024-04-25T11:54:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 5, 2024-04-25T11:54:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 3, 2024-04-25T11:54:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 4, 2024-04-25T11:54:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-04-25T11:54:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 0, 2024-04-25T11:54:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 0, 2024-04-25T11:54:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 2928, 2024-04-25T11:54:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-04-25T11:54:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-25T11:54:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:54:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-25T11:54:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:54:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:54:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:54:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:54:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:54:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:54:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:54:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:54:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:54:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:54:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:54:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:54:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:54:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-25T11:54:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:54:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-25T11:54:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:54:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:54:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:54:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:54:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:54:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:54:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:54:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-25T11:54:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:54:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:54:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:54:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-25T11:54:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-25T11:54:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-25T11:54:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:54:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:54:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:54:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:54:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 189383, 2024-04-25T11:54:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 50, 2024-04-25T11:54:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:54:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-25T11:54:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-25T11:54:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-25T11:54:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:54:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-25T11:54:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:54:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-25T11:54:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 8032, 2024-04-25T11:54:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 167, 2024-04-25T11:54:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:54:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:54:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-25T11:54:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-25T11:54:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 47746, 2024-04-25T11:54:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 220, 2024-04-25T11:54:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-25T11:54:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-25T11:54:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-25T11:54:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-25T11:54:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-04-25T11:54:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-04-25T11:54:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-04-25T11:54:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 2152, 2024-04-25T11:54:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 26, 2024-04-25T11:54:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 4, 2024-04-25T11:54:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 2152, 2024-04-25T11:54:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-04-25T11:54:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2152, 2024-04-25T11:54:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.882986417, 2024-04-25T11:54:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000410309 2024-04-25T11:54:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-04-25T11:54:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-04-25T11:54:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2152, 2024-04-25T11:54:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 2153 2024-04-25T11:54:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 2152, 2024-04-25T11:54:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 2152, 2024-04-25T11:54:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 0, 2024-04-25T11:54:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-04-25T11:54:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-04-25T11:54:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 1373, 2024-04-25T11:54:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 0, 2024-04-25T11:54:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 776, 2024-04-25T11:54:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 0, 2024-04-25T11:54:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 0, 2024-04-25T11:54:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-04-25T11:54:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-04-25T11:54:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-04-25T11:54:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-04-25T11:54:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 0, 2024-04-25T11:54:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-04-25T11:54:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-04-25T11:54:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-04-25T11:54:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 2, 2024-04-25T11:54:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 0, 2024-04-25T11:54:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:54:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-04-25T11:54:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:54:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-04-25T11:54:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-04-25T11:54:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-04-25T11:54:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:54:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-04-25T11:54:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:54:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-04-25T11:54:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-04-25T11:54:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-04-25T11:54:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-04-25T11:54:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-04-25T11:54:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-04-25T11:54:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-04-25T11:54:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-04-25T11:54:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-04-25T11:54:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-04-25T11:54:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-04-25T11:54:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-04-25T11:54:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-04-25T11:54:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-04-25T11:54:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-04-25T11:54:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-04-25T11:54:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-04-25T11:54:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 97, 2024-04-25T11:54:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-04-25T11:54:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 11, 2024-04-25T11:54:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 5, 2024-04-25T11:54:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 5, 2024-04-25T11:54:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 0, 2024-04-25T11:54:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-04-25T11:54:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 0, 2024-04-25T11:54:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 2, 2024-04-25T11:54:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 0 2024-04-25T11:54:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-04-25T11:54:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 0, 2024-04-25T11:54:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:54:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-04-25T11:54:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-04-25T11:54:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-04-25T11:54:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-04-25T11:54:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-04-25T11:54:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-04-25T11:54:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:54:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-04-25T11:54:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-04-25T11:54:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-04-25T11:54:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-04-25T11:54:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-04-25T11:54:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:54:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T11:54:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 2388, 2024-04-25T11:54:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 2089627, 2024-04-25T11:54:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 2388, 2024-04-25T11:54:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T11:54:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T11:54:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2388, 2024-04-25T11:54:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 2089627, 2024-04-25T11:54:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-04-25T11:54:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T11:54:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T11:54:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T11:54:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 2152, 2024-04-25T11:54:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 90620, 2024-04-25T11:54:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2152, 2024-04-25T11:54:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 90620, 2024-04-25T11:54:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-04-25T11:54:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-04-25T11:54:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T11:54:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T11:54:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 2152, 2024-04-25T11:54:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 2152, 2024-04-25T11:54:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2152, 2024-04-25T11:54:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 2152, 2024-04-25T11:54:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:54:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-25T11:54:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T11:54:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T11:54:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T11:54:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T11:54:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-25T11:54:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-25T11:54:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x55a00052a6e0": { 2024-04-25T11:54:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-25T11:54:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T11:54:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T11:54:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T11:54:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T11:54:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-25T11:54:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-25T11:54:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:03.326 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T11:54:03.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:03 smithi083 ceph-mon[29330]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:54:04.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:03 smithi094 ceph-mon[33032]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:54:04.068 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T11:54:04.069 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T11:54:04.069 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-25T11:54:04.069 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-25T11:54:04.069 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T11:54:04.069 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-04-25T11:54:04.069 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T11:54:04.070 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T11:54:04.072 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:03 smithi111 ceph-mon[32784]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:54:04.309 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:54:05.710 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:05 smithi094 ceph-mon[33032]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:54:05.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:05 smithi083 ceph-mon[29330]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:54:05.962 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T11:54:05.962 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:54:05.962 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 48, 2024-04-25T11:54:05.963 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 58, 2024-04-25T11:54:05.963 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 14241, 2024-04-25T11:54:05.963 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 67335, 2024-04-25T11:54:05.963 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 16, 2024-04-25T11:54:05.963 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-04-25T11:54:05.963 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.008888066, 2024-04-25T11:54:05.963 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.001183104, 2024-04-25T11:54:05.963 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.005581028, 2024-04-25T11:54:05.964 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000543933, 2024-04-25T11:54:05.964 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:05.964 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 55, 2024-04-25T11:54:05.964 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.017521403, 2024-04-25T11:54:05.964 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000318570 2024-04-25T11:54:05.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.964 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 37, 2024-04-25T11:54:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000004860, 2024-04-25T11:54:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000131 2024-04-25T11:54:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 6960, 2024-04-25T11:54:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 7312 2024-04-25T11:54:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.965 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:54:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 112, 2024-04-25T11:54:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 263, 2024-04-25T11:54:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 44633, 2024-04-25T11:54:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 482837, 2024-04-25T11:54:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 10, 2024-04-25T11:54:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 6, 2024-04-25T11:54:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.021491941, 2024-04-25T11:54:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.008956178, 2024-04-25T11:54:05.966 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.011592673, 2024-04-25T11:54:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000182462, 2024-04-25T11:54:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 262, 2024-04-25T11:54:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.040617622, 2024-04-25T11:54:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000155029 2024-04-25T11:54:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:05.967 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 174, 2024-04-25T11:54:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000029836, 2024-04-25T11:54:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000171 2024-04-25T11:54:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:54:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 436144 2024-04-25T11:54:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:54:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 597, 2024-04-25T11:54:05.968 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 778, 2024-04-25T11:54:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1738268, 2024-04-25T11:54:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 271999, 2024-04-25T11:54:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 7, 2024-04-25T11:54:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 5, 2024-04-25T11:54:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.097992337, 2024-04-25T11:54:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.040108366, 2024-04-25T11:54:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.056661734, 2024-04-25T11:54:05.969 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.008752713, 2024-04-25T11:54:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 777, 2024-04-25T11:54:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.049460844, 2024-04-25T11:54:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000063656 2024-04-25T11:54:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 832, 2024-04-25T11:54:05.970 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000205342, 2024-04-25T11:54:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000246 2024-04-25T11:54:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1631584, 2024-04-25T11:54:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 132912 2024-04-25T11:54:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-25T11:54:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-04-25T11:54:05.971 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-25T11:54:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-04-25T11:54:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-25T11:54:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-25T11:54:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 56, 2024-04-25T11:54:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.001363175, 2024-04-25T11:54:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000024342 2024-04-25T11:54:05.972 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-04-25T11:54:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-25T11:54:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-25T11:54:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-04-25T11:54:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000005831, 2024-04-25T11:54:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000002915 2024-04-25T11:54:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:05.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-25T11:54:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 7, 2024-04-25T11:54:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 6, 2024-04-25T11:54:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-04-25T11:54:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 6, 2024-04-25T11:54:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.002940086, 2024-04-25T11:54:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000490014 2024-04-25T11:54:05.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-04-25T11:54:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 1, 2024-04-25T11:54:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-04-25T11:54:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-04-25T11:54:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 11, 2024-04-25T11:54:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-04-25T11:54:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-04-25T11:54:05.975 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 11, 2024-04-25T11:54:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 0, 2024-04-25T11:54:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 11, 2024-04-25T11:54:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-04-25T11:54:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 10, 2024-04-25T11:54:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 2, 2024-04-25T11:54:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 0, 2024-04-25T11:54:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 0, 2024-04-25T11:54:05.976 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 2, 2024-04-25T11:54:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 16, 2024-04-25T11:54:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 8, 2024-04-25T11:54:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 1, 2024-04-25T11:54:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 3, 2024-04-25T11:54:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:54:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-04-25T11:54:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-04-25T11:54:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-04-25T11:54:05.977 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-04-25T11:54:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-04-25T11:54:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-04-25T11:54:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-04-25T11:54:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-04-25T11:54:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-04-25T11:54:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:54:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-04-25T11:54:05.978 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 3, 2024-04-25T11:54:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 117, 2024-04-25T11:54:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 0, 2024-04-25T11:54:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:54:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-04-25T11:54:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:54:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:54:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:54:05.979 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:54:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:54:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-04-25T11:54:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:54:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:54:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:54:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:54:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:54:05.980 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:54:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 0, 2024-04-25T11:54:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:54:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 0, 2024-04-25T11:54:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 0 2024-04-25T11:54:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-04-25T11:54:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-04-25T11:54:05.981 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 0, 2024-04-25T11:54:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 0, 2024-04-25T11:54:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 4, 2024-04-25T11:54:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-04-25T11:54:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-04-25T11:54:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-04-25T11:54:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:54:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-04-25T11:54:05.982 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-04-25T11:54:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-04-25T11:54:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-04-25T11:54:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-04-25T11:54:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:54:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:54:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-04-25T11:54:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-04-25T11:54:05.983 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:54:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:54:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:54:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-04-25T11:54:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-04-25T11:54:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:54:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-04-25T11:54:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-04-25T11:54:05.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-04-25T11:54:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-04-25T11:54:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 6, 2024-04-25T11:54:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-04-25T11:54:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-04-25T11:54:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 6, 2024-04-25T11:54:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-04-25T11:54:05.985 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-04-25T11:54:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 2, 2024-04-25T11:54:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-04-25T11:54:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-04-25T11:54:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 2, 2024-04-25T11:54:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 2, 2024-04-25T11:54:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-04-25T11:54:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-04-25T11:54:05.986 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-04-25T11:54:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 4195956, 2024-04-25T11:54:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 0, 2024-04-25T11:54:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-04-25T11:54:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 5, 2024-04-25T11:54:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.015664631, 2024-04-25T11:54:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.003132926 2024-04-25T11:54:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.987 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 0 2024-04-25T11:54:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-04-25T11:54:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 14, 2024-04-25T11:54:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 14, 2024-04-25T11:54:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 0, 2024-04-25T11:54:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 13, 2024-04-25T11:54:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 14, 2024-04-25T11:54:05.988 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 1, 2024-04-25T11:54:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 11, 2024-04-25T11:54:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 13, 2024-04-25T11:54:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 2, 2024-04-25T11:54:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 0, 2024-04-25T11:54:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 4, 2024-04-25T11:54:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 4, 2024-04-25T11:54:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 54972, 2024-04-25T11:54:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 223608 2024-04-25T11:54:05.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-04-25T11:54:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 13, 2024-04-25T11:54:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 0, 2024-04-25T11:54:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 7, 2024-04-25T11:54:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 8, 2024-04-25T11:54:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 1, 2024-04-25T11:54:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-04-25T11:54:05.990 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-04-25T11:54:05.991 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2, 2024-04-25T11:54:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.001462804, 2024-04-25T11:54:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000731402 2024-04-25T11:54:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-04-25T11:54:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.992 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-04-25T11:54:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-04-25T11:54:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 4, 2024-04-25T11:54:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.001477282, 2024-04-25T11:54:05.993 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000369320 2024-04-25T11:54:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-04-25T11:54:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-04-25T11:54:05.994 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-04-25T11:54:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.995 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-04-25T11:54:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-04-25T11:54:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:05.996 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-04-25T11:54:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-04-25T11:54:05.997 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:05.998 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-04-25T11:54:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-04-25T11:54:05.999 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-04-25T11:54:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.000 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-04-25T11:54:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-04-25T11:54:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.001 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-04-25T11:54:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.002 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-04-25T11:54:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-04-25T11:54:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.003 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-04-25T11:54:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.004 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-04-25T11:54:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-04-25T11:54:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-04-25T11:54:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-04-25T11:54:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.006 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-04-25T11:54:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.007 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-04-25T11:54:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-04-25T11:54:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-04-25T11:54:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:54:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:54:06.009 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-04-25T11:54:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-04-25T11:54:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 0, 2024-04-25T11:54:06.010 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 2, 2024-04-25T11:54:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 2, 2024-04-25T11:54:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 0, 2024-04-25T11:54:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-04-25T11:54:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 5, 2024-04-25T11:54:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 2, 2024-04-25T11:54:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 88, 2024-04-25T11:54:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-04-25T11:54:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.011 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-25T11:54:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:54:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-25T11:54:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:54:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:54:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:54:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:54:06.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:54:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:54:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:54:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:54:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:54:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:54:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:54:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:54:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:54:06.013 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-25T11:54:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:54:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-25T11:54:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:54:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:54:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:54:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:54:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:54:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:54:06.014 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:54:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-25T11:54:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:54:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:54:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:54:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-25T11:54:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-25T11:54:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-25T11:54:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:54:06.015 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:54:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:54:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:54:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 255543, 2024-04-25T11:54:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 89, 2024-04-25T11:54:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:54:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-25T11:54:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-25T11:54:06.016 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-25T11:54:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:54:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-25T11:54:06.017 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:54:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-25T11:54:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 8032, 2024-04-25T11:54:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 167, 2024-04-25T11:54:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:54:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:54:06.018 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-25T11:54:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-25T11:54:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 56576, 2024-04-25T11:54:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 228, 2024-04-25T11:54:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-25T11:54:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-25T11:54:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-25T11:54:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-25T11:54:06.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-04-25T11:54:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-04-25T11:54:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-04-25T11:54:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 43, 2024-04-25T11:54:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 12099, 2024-04-25T11:54:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 29, 2024-04-25T11:54:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 43, 2024-04-25T11:54:06.020 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-04-25T11:54:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 43, 2024-04-25T11:54:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.225996424, 2024-04-25T11:54:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.005255730 2024-04-25T11:54:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-04-25T11:54:06.021 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-04-25T11:54:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 43, 2024-04-25T11:54:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 67 2024-04-25T11:54:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 43, 2024-04-25T11:54:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 0, 2024-04-25T11:54:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 43, 2024-04-25T11:54:06.022 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-04-25T11:54:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-04-25T11:54:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 0, 2024-04-25T11:54:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 10, 2024-04-25T11:54:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 0, 2024-04-25T11:54:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 5, 2024-04-25T11:54:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 7, 2024-04-25T11:54:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-04-25T11:54:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-04-25T11:54:06.023 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-04-25T11:54:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-04-25T11:54:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 6, 2024-04-25T11:54:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-04-25T11:54:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-04-25T11:54:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-04-25T11:54:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 0, 2024-04-25T11:54:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 20, 2024-04-25T11:54:06.024 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:54:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-04-25T11:54:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:54:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-04-25T11:54:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-04-25T11:54:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-04-25T11:54:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:54:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-04-25T11:54:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:54:06.025 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-04-25T11:54:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-04-25T11:54:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-04-25T11:54:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-04-25T11:54:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-04-25T11:54:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-04-25T11:54:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-04-25T11:54:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-04-25T11:54:06.026 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-04-25T11:54:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-04-25T11:54:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-04-25T11:54:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-04-25T11:54:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-04-25T11:54:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-04-25T11:54:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-04-25T11:54:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-04-25T11:54:06.027 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-04-25T11:54:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 97, 2024-04-25T11:54:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-04-25T11:54:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 11, 2024-04-25T11:54:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 8, 2024-04-25T11:54:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 8, 2024-04-25T11:54:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 0, 2024-04-25T11:54:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-04-25T11:54:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 17, 2024-04-25T11:54:06.028 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 0, 2024-04-25T11:54:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 2 2024-04-25T11:54:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-04-25T11:54:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 1, 2024-04-25T11:54:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:54:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-04-25T11:54:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-04-25T11:54:06.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-04-25T11:54:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-04-25T11:54:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-04-25T11:54:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-04-25T11:54:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:54:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-04-25T11:54:06.030 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-04-25T11:54:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-04-25T11:54:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-04-25T11:54:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-04-25T11:54:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:54:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T11:54:06.031 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 753, 2024-04-25T11:54:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1712513, 2024-04-25T11:54:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 753, 2024-04-25T11:54:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T11:54:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T11:54:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 753, 2024-04-25T11:54:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1712513, 2024-04-25T11:54:06.032 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-04-25T11:54:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:06.033 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T11:54:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T11:54:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T11:54:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 43, 2024-04-25T11:54:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 12099, 2024-04-25T11:54:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 43, 2024-04-25T11:54:06.034 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 12099, 2024-04-25T11:54:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-04-25T11:54:06.035 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-04-25T11:54:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T11:54:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T11:54:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 43, 2024-04-25T11:54:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 43, 2024-04-25T11:54:06.036 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 43, 2024-04-25T11:54:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 43, 2024-04-25T11:54:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:54:06.037 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-25T11:54:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T11:54:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T11:54:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T11:54:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T11:54:06.038 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-25T11:54:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-25T11:54:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:06.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x55e361d54500": { 2024-04-25T11:54:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-25T11:54:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 6, 2024-04-25T11:54:06.040 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1652, 2024-04-25T11:54:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 6, 2024-04-25T11:54:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T11:54:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T11:54:06.041 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 5, 2024-04-25T11:54:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1652, 2024-04-25T11:54:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:06.042 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:06.042 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T11:54:06.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:05 smithi111 ceph-mon[32784]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:54:06.777 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T11:54:06.777 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T11:54:06.777 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-25T11:54:06.777 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T11:54:06.778 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T11:54:06.778 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-25T11:54:06.778 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T11:54:06.778 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T11:54:07.016 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T11:54:07.774 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:07 smithi094 ceph-mon[33032]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:54:07.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:07 smithi083 ceph-mon[29330]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:54:08.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:07 smithi111 ceph-mon[32784]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-25T11:54:08.691 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T11:54:08.692 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:54:08.692 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 974, 2024-04-25T11:54:08.692 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 962, 2024-04-25T11:54:08.692 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1865904, 2024-04-25T11:54:08.692 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 365678, 2024-04-25T11:54:08.692 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-04-25T11:54:08.692 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-04-25T11:54:08.692 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.126435473, 2024-04-25T11:54:08.693 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.046864937, 2024-04-25T11:54:08.693 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.055041167, 2024-04-25T11:54:08.693 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.025866749, 2024-04-25T11:54:08.693 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:08.693 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 961, 2024-04-25T11:54:08.693 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.049510403, 2024-04-25T11:54:08.693 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000051519 2024-04-25T11:54:08.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.694 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:08.694 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.694 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.694 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.694 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1618272, 2024-04-25T11:54:08.694 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 131312 2024-04-25T11:54:08.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.695 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:54:08.695 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 4, 2024-04-25T11:54:08.695 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 162, 2024-04-25T11:54:08.695 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 876, 2024-04-25T11:54:08.695 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 440954, 2024-04-25T11:54:08.695 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-04-25T11:54:08.695 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 4, 2024-04-25T11:54:08.695 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.010696319, 2024-04-25T11:54:08.695 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.006872058, 2024-04-25T11:54:08.696 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.001940328, 2024-04-25T11:54:08.696 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000047870, 2024-04-25T11:54:08.696 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:08.696 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 159, 2024-04-25T11:54:08.696 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.014200972, 2024-04-25T11:54:08.696 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000089314 2024-04-25T11:54:08.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.696 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:08.697 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.697 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.697 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.697 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:54:08.697 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 440256 2024-04-25T11:54:08.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.697 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:54:08.697 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1396, 2024-04-25T11:54:08.698 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1396, 2024-04-25T11:54:08.698 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 359465, 2024-04-25T11:54:08.698 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 443111, 2024-04-25T11:54:08.698 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-04-25T11:54:08.698 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-04-25T11:54:08.699 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.086457977, 2024-04-25T11:54:08.699 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.020958753, 2024-04-25T11:54:08.699 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.024542125, 2024-04-25T11:54:08.700 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.035324360, 2024-04-25T11:54:08.700 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:08.700 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1395, 2024-04-25T11:54:08.700 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.037725371, 2024-04-25T11:54:08.700 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000027043 2024-04-25T11:54:08.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.700 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:08.700 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.701 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.701 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.701 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T11:54:08.701 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T11:54:08.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.701 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-25T11:54:08.701 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-04-25T11:54:08.701 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-25T11:54:08.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.702 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-04-25T11:54:08.702 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-25T11:54:08.702 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-25T11:54:08.702 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3568, 2024-04-25T11:54:08.702 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.081847188, 2024-04-25T11:54:08.702 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000022939 2024-04-25T11:54:08.702 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:08.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.703 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-04-25T11:54:08.703 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-25T11:54:08.703 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-25T11:54:08.703 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.703 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.703 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.703 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:08.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.704 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-25T11:54:08.704 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 0, 2024-04-25T11:54:08.704 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 0, 2024-04-25T11:54:08.704 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-04-25T11:54:08.704 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.704 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.704 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.705 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-04-25T11:54:08.705 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 0, 2024-04-25T11:54:08.705 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-04-25T11:54:08.705 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-04-25T11:54:08.705 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 0, 2024-04-25T11:54:08.705 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-04-25T11:54:08.705 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-04-25T11:54:08.705 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 0, 2024-04-25T11:54:08.705 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 0, 2024-04-25T11:54:08.706 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 0, 2024-04-25T11:54:08.706 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-04-25T11:54:08.706 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 0, 2024-04-25T11:54:08.706 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 0, 2024-04-25T11:54:08.706 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 0, 2024-04-25T11:54:08.706 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 0, 2024-04-25T11:54:08.706 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 1, 2024-04-25T11:54:08.706 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 0, 2024-04-25T11:54:08.707 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 0, 2024-04-25T11:54:08.707 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 0, 2024-04-25T11:54:08.707 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 0, 2024-04-25T11:54:08.707 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:54:08.707 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-04-25T11:54:08.707 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-04-25T11:54:08.707 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-04-25T11:54:08.707 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-04-25T11:54:08.707 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-04-25T11:54:08.708 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-04-25T11:54:08.708 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-04-25T11:54:08.708 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-04-25T11:54:08.708 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-04-25T11:54:08.708 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:54:08.708 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-04-25T11:54:08.708 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 0, 2024-04-25T11:54:08.708 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 0, 2024-04-25T11:54:08.709 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 0, 2024-04-25T11:54:08.709 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:54:08.709 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-04-25T11:54:08.709 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:54:08.709 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:54:08.709 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:54:08.709 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:54:08.709 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:54:08.710 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-04-25T11:54:08.710 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:54:08.710 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:54:08.711 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:54:08.711 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:54:08.711 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:54:08.711 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:54:08.711 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 0, 2024-04-25T11:54:08.711 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:54:08.711 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 0, 2024-04-25T11:54:08.711 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 0 2024-04-25T11:54:08.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.712 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-04-25T11:54:08.712 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-04-25T11:54:08.712 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 0, 2024-04-25T11:54:08.712 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 0, 2024-04-25T11:54:08.712 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 0, 2024-04-25T11:54:08.712 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-04-25T11:54:08.712 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-04-25T11:54:08.712 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-04-25T11:54:08.713 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:54:08.713 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-04-25T11:54:08.713 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-04-25T11:54:08.713 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-04-25T11:54:08.713 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-04-25T11:54:08.713 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-04-25T11:54:08.713 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:54:08.713 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:54:08.714 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-04-25T11:54:08.714 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-04-25T11:54:08.714 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:54:08.714 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:54:08.714 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:54:08.714 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-04-25T11:54:08.714 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-04-25T11:54:08.714 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:54:08.715 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-04-25T11:54:08.715 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-04-25T11:54:08.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.715 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-04-25T11:54:08.715 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-04-25T11:54:08.715 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 0, 2024-04-25T11:54:08.715 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-04-25T11:54:08.715 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-04-25T11:54:08.715 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 6, 2024-04-25T11:54:08.716 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-04-25T11:54:08.716 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-04-25T11:54:08.716 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 0, 2024-04-25T11:54:08.716 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-04-25T11:54:08.716 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-04-25T11:54:08.716 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 2, 2024-04-25T11:54:08.716 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 2, 2024-04-25T11:54:08.716 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-04-25T11:54:08.716 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-04-25T11:54:08.717 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4194304, 2024-04-25T11:54:08.717 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 4195956, 2024-04-25T11:54:08.717 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 4195815, 2024-04-25T11:54:08.717 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-04-25T11:54:08.717 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.717 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.717 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.718 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 6 2024-04-25T11:54:08.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.718 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-04-25T11:54:08.718 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 3, 2024-04-25T11:54:08.718 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 3, 2024-04-25T11:54:08.718 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 0, 2024-04-25T11:54:08.718 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 1, 2024-04-25T11:54:08.718 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 1, 2024-04-25T11:54:08.718 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 0, 2024-04-25T11:54:08.719 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 0, 2024-04-25T11:54:08.719 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 0, 2024-04-25T11:54:08.719 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 0, 2024-04-25T11:54:08.719 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 0, 2024-04-25T11:54:08.719 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 0, 2024-04-25T11:54:08.719 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 0, 2024-04-25T11:54:08.719 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 50400, 2024-04-25T11:54:08.719 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 215416 2024-04-25T11:54:08.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.720 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-04-25T11:54:08.720 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 0, 2024-04-25T11:54:08.720 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 0, 2024-04-25T11:54:08.720 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 0, 2024-04-25T11:54:08.720 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 0, 2024-04-25T11:54:08.720 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 0, 2024-04-25T11:54:08.720 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-04-25T11:54:08.720 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.721 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.721 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.721 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-04-25T11:54:08.721 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.721 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.721 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.722 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-04-25T11:54:08.722 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.722 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.722 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.722 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-04-25T11:54:08.722 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.722 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.723 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.723 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-04-25T11:54:08.723 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.723 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.723 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.723 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-04-25T11:54:08.724 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.724 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.724 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.724 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-04-25T11:54:08.724 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.724 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.725 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.725 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-04-25T11:54:08.725 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.725 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.725 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.725 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-04-25T11:54:08.725 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.726 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.726 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.727 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-04-25T11:54:08.727 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.727 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.727 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.727 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-04-25T11:54:08.727 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.728 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.728 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.728 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-04-25T11:54:08.728 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.729 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.729 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.729 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-04-25T11:54:08.729 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.730 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.730 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.730 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-04-25T11:54:08.730 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.730 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.730 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.731 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-04-25T11:54:08.731 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.731 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.731 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.731 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-04-25T11:54:08.731 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.731 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.732 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.732 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-04-25T11:54:08.732 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.732 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.732 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.732 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-04-25T11:54:08.732 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.733 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.733 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.733 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-04-25T11:54:08.733 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.733 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.733 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.734 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-04-25T11:54:08.734 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.734 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.734 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.734 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-04-25T11:54:08.734 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.734 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.735 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.735 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-04-25T11:54:08.735 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.735 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.735 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.735 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-04-25T11:54:08.735 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.736 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.736 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.736 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-04-25T11:54:08.736 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.736 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.736 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.737 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-04-25T11:54:08.737 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.737 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.737 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.737 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-04-25T11:54:08.737 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.737 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.737 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.738 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-04-25T11:54:08.738 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.738 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.738 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.738 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-04-25T11:54:08.738 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.739 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.739 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.739 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-04-25T11:54:08.739 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.739 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.739 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.740 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:54:08.740 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:54:08.740 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-04-25T11:54:08.740 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.740 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.740 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.740 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:08.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.741 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-04-25T11:54:08.741 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 0, 2024-04-25T11:54:08.741 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 2, 2024-04-25T11:54:08.741 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 2, 2024-04-25T11:54:08.741 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 1, 2024-04-25T11:54:08.741 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-04-25T11:54:08.741 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 0, 2024-04-25T11:54:08.741 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 0, 2024-04-25T11:54:08.741 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 1240, 2024-04-25T11:54:08.742 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-04-25T11:54:08.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.742 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-25T11:54:08.742 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:54:08.742 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-25T11:54:08.742 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:54:08.742 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:54:08.742 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:54:08.743 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:54:08.743 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:54:08.743 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:54:08.743 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:54:08.743 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:54:08.743 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:54:08.743 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:54:08.743 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:54:08.743 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:54:08.744 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:54:08.744 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-25T11:54:08.744 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:54:08.744 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-25T11:54:08.744 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:54:08.744 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:54:08.744 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:54:08.744 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:54:08.745 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:54:08.745 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:54:08.745 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:54:08.745 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-25T11:54:08.745 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:54:08.745 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:54:08.745 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:54:08.745 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-25T11:54:08.745 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-25T11:54:08.746 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-25T11:54:08.746 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:54:08.746 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:54:08.746 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:54:08.746 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:54:08.746 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 181330, 2024-04-25T11:54:08.746 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 48, 2024-04-25T11:54:08.747 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:54:08.747 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-25T11:54:08.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-25T11:54:08.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-25T11:54:08.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:54:08.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-25T11:54:08.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:54:08.747 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-25T11:54:08.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 8032, 2024-04-25T11:54:08.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 167, 2024-04-25T11:54:08.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:54:08.748 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:54:08.748 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-25T11:54:08.748 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-25T11:54:08.748 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 8512, 2024-04-25T11:54:08.748 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 13, 2024-04-25T11:54:08.748 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-25T11:54:08.749 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-25T11:54:08.749 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-25T11:54:08.749 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-25T11:54:08.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.749 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-04-25T11:54:08.749 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-04-25T11:54:08.749 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-04-25T11:54:08.750 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 2131, 2024-04-25T11:54:08.750 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 21, 2024-04-25T11:54:08.750 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 3, 2024-04-25T11:54:08.750 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 2131, 2024-04-25T11:54:08.750 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-04-25T11:54:08.750 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2131, 2024-04-25T11:54:08.750 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 1.160982112, 2024-04-25T11:54:08.751 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000544806 2024-04-25T11:54:08.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.752 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-04-25T11:54:08.752 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-04-25T11:54:08.753 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2131, 2024-04-25T11:54:08.753 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 2132 2024-04-25T11:54:08.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.753 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 2131, 2024-04-25T11:54:08.753 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 2131, 2024-04-25T11:54:08.753 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 0, 2024-04-25T11:54:08.753 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-04-25T11:54:08.754 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-04-25T11:54:08.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 1385, 2024-04-25T11:54:08.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 0, 2024-04-25T11:54:08.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 744, 2024-04-25T11:54:08.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 0, 2024-04-25T11:54:08.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 0, 2024-04-25T11:54:08.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-04-25T11:54:08.754 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-04-25T11:54:08.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-04-25T11:54:08.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-04-25T11:54:08.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 0, 2024-04-25T11:54:08.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-04-25T11:54:08.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-04-25T11:54:08.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-04-25T11:54:08.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 1, 2024-04-25T11:54:08.755 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 0, 2024-04-25T11:54:08.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:54:08.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-04-25T11:54:08.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:54:08.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-04-25T11:54:08.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-04-25T11:54:08.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-04-25T11:54:08.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:54:08.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-04-25T11:54:08.756 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:54:08.757 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-04-25T11:54:08.757 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-04-25T11:54:08.757 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-04-25T11:54:08.757 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-04-25T11:54:08.757 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-04-25T11:54:08.757 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-04-25T11:54:08.757 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-04-25T11:54:08.757 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-04-25T11:54:08.758 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-04-25T11:54:08.758 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-04-25T11:54:08.758 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-04-25T11:54:08.758 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-04-25T11:54:08.758 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-04-25T11:54:08.758 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-04-25T11:54:08.759 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-04-25T11:54:08.759 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-04-25T11:54:08.759 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-04-25T11:54:08.759 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 97, 2024-04-25T11:54:08.759 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-04-25T11:54:08.759 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 11, 2024-04-25T11:54:08.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 4, 2024-04-25T11:54:08.759 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 4, 2024-04-25T11:54:08.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 0, 2024-04-25T11:54:08.760 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-04-25T11:54:08.760 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 0, 2024-04-25T11:54:08.760 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 2, 2024-04-25T11:54:08.760 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 0 2024-04-25T11:54:08.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.760 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-04-25T11:54:08.760 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 0, 2024-04-25T11:54:08.760 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:54:08.761 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-04-25T11:54:08.761 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-04-25T11:54:08.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.761 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-04-25T11:54:08.761 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-04-25T11:54:08.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.761 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-04-25T11:54:08.761 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-04-25T11:54:08.762 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:54:08.762 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-04-25T11:54:08.762 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-04-25T11:54:08.762 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-04-25T11:54:08.762 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-04-25T11:54:08.762 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-04-25T11:54:08.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.762 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:54:08.763 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:08.763 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T11:54:08.763 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:08.763 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 2370, 2024-04-25T11:54:08.763 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1997870, 2024-04-25T11:54:08.763 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:08.763 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 2370, 2024-04-25T11:54:08.763 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T11:54:08.764 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T11:54:08.764 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2370, 2024-04-25T11:54:08.764 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1997870, 2024-04-25T11:54:08.764 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:08.764 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.764 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.764 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.764 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:08.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.765 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-04-25T11:54:08.765 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:08.765 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T11:54:08.765 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:08.765 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T11:54:08.765 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T11:54:08.765 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:08.766 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:08.766 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 2131, 2024-04-25T11:54:08.766 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 1657, 2024-04-25T11:54:08.766 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2131, 2024-04-25T11:54:08.766 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1657, 2024-04-25T11:54:08.766 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:08.766 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.767 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.767 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.767 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:08.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.767 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-04-25T11:54:08.767 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:08.767 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-04-25T11:54:08.767 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:08.768 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T11:54:08.768 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T11:54:08.768 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:08.768 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:08.768 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 2131, 2024-04-25T11:54:08.768 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 2131, 2024-04-25T11:54:08.768 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2131, 2024-04-25T11:54:08.768 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 2131, 2024-04-25T11:54:08.769 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:08.769 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.769 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.769 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.769 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:08.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.769 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:54:08.769 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:08.770 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-25T11:54:08.770 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:08.770 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T11:54:08.770 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T11:54:08.770 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:08.770 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:08.770 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T11:54:08.770 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T11:54:08.771 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-25T11:54:08.771 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-25T11:54:08.771 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:08.771 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.771 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.771 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.771 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:08.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T11:54:08.771 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x55f5447926e0": { 2024-04-25T11:54:08.772 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T11:54:08.772 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-25T11:54:08.772 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T11:54:08.772 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T11:54:08.772 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T11:54:08.772 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:08.772 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:08.772 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T11:54:08.773 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T11:54:08.773 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-25T11:54:08.773 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-25T11:54:08.773 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T11:54:08.773 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T11:54:08.773 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T11:54:08.773 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T11:54:08.773 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:08.773 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T11:54:08.774 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T11:54:09.491 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-25T11:54:09.491 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-25T11:54:09.491 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-04-25T11:54:09.491 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-25T11:54:09.492 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T11:54:09.492 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-04-25T11:54:09.492 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T11:54:09.492 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T11:54:09.742 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:54:09.774 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:09 smithi094 ceph-mon[33032]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:54:09.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:09 smithi111 ceph-mon[32784]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:54:09.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:09 smithi083 ceph-mon[29330]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:54:11.481 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-25T11:54:11.481 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:54:11.481 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 86, 2024-04-25T11:54:11.481 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 90, 2024-04-25T11:54:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 29719, 2024-04-25T11:54:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 60160, 2024-04-25T11:54:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 13, 2024-04-25T11:54:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 4, 2024-04-25T11:54:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.010965537, 2024-04-25T11:54:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.003046521, 2024-04-25T11:54:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.006185224, 2024-04-25T11:54:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000579812, 2024-04-25T11:54:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 88, 2024-04-25T11:54:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.022893128, 2024-04-25T11:54:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000260149 2024-04-25T11:54:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 100, 2024-04-25T11:54:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000022415, 2024-04-25T11:54:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000224 2024-04-25T11:54:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1856, 2024-04-25T11:54:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 1952 2024-04-25T11:54:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:54:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 340, 2024-04-25T11:54:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 448, 2024-04-25T11:54:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 1666251, 2024-04-25T11:54:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 506496, 2024-04-25T11:54:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 10, 2024-04-25T11:54:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 6, 2024-04-25T11:54:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.061302657, 2024-04-25T11:54:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.021183297, 2024-04-25T11:54:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.044432938, 2024-04-25T11:54:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.005617994, 2024-04-25T11:54:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 448, 2024-04-25T11:54:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.040610437, 2024-04-25T11:54:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000090648 2024-04-25T11:54:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 99, 2024-04-25T11:54:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000025796, 2024-04-25T11:54:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000260 2024-04-25T11:54:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1641664, 2024-04-25T11:54:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 450112 2024-04-25T11:54:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:54:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 352, 2024-04-25T11:54:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 594, 2024-04-25T11:54:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 111257, 2024-04-25T11:54:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 382609, 2024-04-25T11:54:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 11, 2024-04-25T11:54:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 7, 2024-04-25T11:54:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.058949949, 2024-04-25T11:54:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.023879666, 2024-04-25T11:54:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.027670801, 2024-04-25T11:54:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.004748965, 2024-04-25T11:54:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 591, 2024-04-25T11:54:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.037568327, 2024-04-25T11:54:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000063567 2024-04-25T11:54:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 892, 2024-04-25T11:54:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000264987, 2024-04-25T11:54:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000297 2024-04-25T11:54:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 2816, 2024-04-25T11:54:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 133312 2024-04-25T11:54:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-04-25T11:54:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-04-25T11:54:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-04-25T11:54:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-04-25T11:54:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-25T11:54:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-25T11:54:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 60, 2024-04-25T11:54:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.001709822, 2024-04-25T11:54:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000028497 2024-04-25T11:54:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.493 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-04-25T11:54:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-25T11:54:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-25T11:54:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-04-25T11:54:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000008617, 2024-04-25T11:54:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000004308 2024-04-25T11:54:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-04-25T11:54:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 0, 2024-04-25T11:54:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 0, 2024-04-25T11:54:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-04-25T11:54:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-04-25T11:54:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2024-04-25T11:54:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-04-25T11:54:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-04-25T11:54:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 11, 2024-04-25T11:54:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-04-25T11:54:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-04-25T11:54:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 10, 2024-04-25T11:54:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 0, 2024-04-25T11:54:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 10, 2024-04-25T11:54:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-04-25T11:54:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 10, 2024-04-25T11:54:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-04-25T11:54:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2024-04-25T11:54:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2024-04-25T11:54:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 1, 2024-04-25T11:54:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 0, 2024-04-25T11:54:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 0, 2024-04-25T11:54:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-04-25T11:54:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 0, 2024-04-25T11:54:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:54:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-04-25T11:54:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-04-25T11:54:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-04-25T11:54:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-04-25T11:54:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-04-25T11:54:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-04-25T11:54:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-04-25T11:54:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-04-25T11:54:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-04-25T11:54:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:54:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-04-25T11:54:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 3, 2024-04-25T11:54:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 117, 2024-04-25T11:54:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2024-04-25T11:54:11.500 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:54:11.500 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-04-25T11:54:11.500 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:54:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:54:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:54:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:54:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:54:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-04-25T11:54:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:54:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:54:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:54:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:54:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:54:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:54:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2024-04-25T11:54:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:54:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-04-25T11:54:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2024-04-25T11:54:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-04-25T11:54:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-04-25T11:54:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-04-25T11:54:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-04-25T11:54:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 1, 2024-04-25T11:54:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-04-25T11:54:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-04-25T11:54:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-04-25T11:54:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:54:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-04-25T11:54:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-04-25T11:54:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-04-25T11:54:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-04-25T11:54:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-04-25T11:54:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:54:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:54:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-04-25T11:54:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-04-25T11:54:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:54:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:54:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:54:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-04-25T11:54:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-04-25T11:54:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:54:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-04-25T11:54:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-04-25T11:54:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-04-25T11:54:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-04-25T11:54:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 9, 2024-04-25T11:54:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-04-25T11:54:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-04-25T11:54:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 9, 2024-04-25T11:54:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-04-25T11:54:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-04-25T11:54:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 2, 2024-04-25T11:54:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-04-25T11:54:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-04-25T11:54:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 2, 2024-04-25T11:54:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 2, 2024-04-25T11:54:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-04-25T11:54:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-04-25T11:54:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4194304, 2024-04-25T11:54:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 4196879, 2024-04-25T11:54:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 0, 2024-04-25T11:54:11.509 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-04-25T11:54:11.509 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 8, 2024-04-25T11:54:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.019429390, 2024-04-25T11:54:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.002428673 2024-04-25T11:54:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 0 2024-04-25T11:54:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-04-25T11:54:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 13, 2024-04-25T11:54:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 13, 2024-04-25T11:54:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 0, 2024-04-25T11:54:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 11, 2024-04-25T11:54:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 12, 2024-04-25T11:54:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 1, 2024-04-25T11:54:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 10, 2024-04-25T11:54:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 10, 2024-04-25T11:54:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 0, 2024-04-25T11:54:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2024-04-25T11:54:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 0, 2024-04-25T11:54:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 0, 2024-04-25T11:54:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 51968, 2024-04-25T11:54:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 223608 2024-04-25T11:54:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-04-25T11:54:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 0, 2024-04-25T11:54:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-04-25T11:54:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 0, 2024-04-25T11:54:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 21, 2024-04-25T11:54:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 0, 2024-04-25T11:54:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-04-25T11:54:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-04-25T11:54:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.516 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.516 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-04-25T11:54:11.516 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.516 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.516 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-04-25T11:54:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-04-25T11:54:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-04-25T11:54:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-04-25T11:54:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-04-25T11:54:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-04-25T11:54:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-04-25T11:54:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-04-25T11:54:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-04-25T11:54:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.523 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.523 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.524 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.524 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-04-25T11:54:11.524 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.524 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.524 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.524 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.524 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-04-25T11:54:11.524 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.524 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.525 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.525 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.525 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-04-25T11:54:11.525 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.525 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.525 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.525 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.526 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-04-25T11:54:11.526 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.526 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.526 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.526 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.526 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-04-25T11:54:11.526 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.526 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.527 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.527 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.527 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-04-25T11:54:11.527 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.527 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.527 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.527 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.527 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-04-25T11:54:11.527 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.528 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.528 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.528 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.528 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-04-25T11:54:11.528 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.528 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.528 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.528 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.528 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-04-25T11:54:11.529 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.529 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.529 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.529 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.529 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-04-25T11:54:11.529 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.529 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.529 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.530 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.530 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-04-25T11:54:11.530 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.530 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.530 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.530 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.530 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-04-25T11:54:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-04-25T11:54:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-04-25T11:54:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-04-25T11:54:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-04-25T11:54:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-04-25T11:54:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:54:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:54:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-04-25T11:54:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-04-25T11:54:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 1, 2024-04-25T11:54:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 4, 2024-04-25T11:54:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 3, 2024-04-25T11:54:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 1, 2024-04-25T11:54:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-04-25T11:54:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 0, 2024-04-25T11:54:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 0, 2024-04-25T11:54:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 610, 2024-04-25T11:54:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-04-25T11:54:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-04-25T11:54:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:54:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-04-25T11:54:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:54:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:54:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:54:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:54:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:54:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:54:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:54:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:54:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:54:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:54:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:54:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:54:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:54:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-04-25T11:54:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:54:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-04-25T11:54:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:54:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:54:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:54:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:54:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:54:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:54:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:54:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-04-25T11:54:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:54:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:54:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:54:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-04-25T11:54:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-04-25T11:54:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-04-25T11:54:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:54:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:54:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:54:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:54:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 230535, 2024-04-25T11:54:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 80, 2024-04-25T11:54:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:54:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-04-25T11:54:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-04-25T11:54:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-04-25T11:54:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:54:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-04-25T11:54:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:54:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-04-25T11:54:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 8152, 2024-04-25T11:54:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 170, 2024-04-25T11:54:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:54:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:54:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-04-25T11:54:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-04-25T11:54:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 50872, 2024-04-25T11:54:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 198, 2024-04-25T11:54:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-04-25T11:54:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-04-25T11:54:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-04-25T11:54:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-04-25T11:54:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-04-25T11:54:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-04-25T11:54:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-04-25T11:54:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 46, 2024-04-25T11:54:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 13057, 2024-04-25T11:54:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 33, 2024-04-25T11:54:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 46, 2024-04-25T11:54:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-04-25T11:54:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 46, 2024-04-25T11:54:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.214995958, 2024-04-25T11:54:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.004673825 2024-04-25T11:54:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-04-25T11:54:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-04-25T11:54:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 46, 2024-04-25T11:54:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 68 2024-04-25T11:54:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 46, 2024-04-25T11:54:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 0, 2024-04-25T11:54:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 46, 2024-04-25T11:54:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-04-25T11:54:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-04-25T11:54:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 0, 2024-04-25T11:54:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 10, 2024-04-25T11:54:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 0, 2024-04-25T11:54:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 8, 2024-04-25T11:54:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 9, 2024-04-25T11:54:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-04-25T11:54:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-04-25T11:54:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-04-25T11:54:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-04-25T11:54:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 6, 2024-04-25T11:54:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-04-25T11:54:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-04-25T11:54:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-04-25T11:54:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 0, 2024-04-25T11:54:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 20, 2024-04-25T11:54:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:54:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-04-25T11:54:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:54:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-04-25T11:54:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-04-25T11:54:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-04-25T11:54:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:54:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-04-25T11:54:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:54:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-04-25T11:54:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-04-25T11:54:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-04-25T11:54:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-04-25T11:54:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-04-25T11:54:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-04-25T11:54:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-04-25T11:54:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-04-25T11:54:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-04-25T11:54:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-04-25T11:54:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-04-25T11:54:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-04-25T11:54:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-04-25T11:54:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-04-25T11:54:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-04-25T11:54:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-04-25T11:54:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-04-25T11:54:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 98, 2024-04-25T11:54:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-04-25T11:54:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 12, 2024-04-25T11:54:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 8, 2024-04-25T11:54:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 8, 2024-04-25T11:54:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-04-25T11:54:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-04-25T11:54:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 14, 2024-04-25T11:54:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 0, 2024-04-25T11:54:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 1 2024-04-25T11:54:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-04-25T11:54:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 1, 2024-04-25T11:54:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:54:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-04-25T11:54:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-04-25T11:54:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-04-25T11:54:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2024-04-25T11:54:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-04-25T11:54:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-04-25T11:54:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:54:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-04-25T11:54:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-04-25T11:54:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-04-25T11:54:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-04-25T11:54:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-04-25T11:54:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:54:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-25T11:54:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 774, 2024-04-25T11:54:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1720087, 2024-04-25T11:54:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 774, 2024-04-25T11:54:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-25T11:54:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-25T11:54:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 774, 2024-04-25T11:54:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1720087, 2024-04-25T11:54:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-04-25T11:54:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-25T11:54:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-25T11:54:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-25T11:54:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 46, 2024-04-25T11:54:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 13057, 2024-04-25T11:54:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 46, 2024-04-25T11:54:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 13057, 2024-04-25T11:54:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-04-25T11:54:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-04-25T11:54:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-25T11:54:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-25T11:54:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 46, 2024-04-25T11:54:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 46, 2024-04-25T11:54:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 46, 2024-04-25T11:54:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 46, 2024-04-25T11:54:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:54:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-25T11:54:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-25T11:54:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-25T11:54:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-25T11:54:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-25T11:54:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-04-25T11:54:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-04-25T11:54:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x55b25e80a500": { 2024-04-25T11:54:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-25T11:54:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 9, 2024-04-25T11:54:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 2575, 2024-04-25T11:54:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 9, 2024-04-25T11:54:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-25T11:54:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-25T11:54:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 8, 2024-04-25T11:54:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 2575, 2024-04-25T11:54:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:11.569 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-25T11:54:11.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:11 smithi111 ceph-mon[32784]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:54:11.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:11 smithi111 ceph-mon[32784]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T11:54:11.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:11 smithi083 ceph-mon[29330]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:54:11.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:11 smithi083 ceph-mon[29330]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T11:54:11.964 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:11 smithi094 ceph-mon[33032]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-25T11:54:11.965 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:11 smithi094 ceph-mon[33032]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T11:54:12.271 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T11:54:12.271 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T11:54:12.271 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-25T11:54:12.271 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T11:54:12.271 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T11:54:12.271 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-25T11:54:12.271 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T11:54:12.272 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T11:54:12.511 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:54:13.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:13 smithi111 ceph-mon[32784]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:54:13.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:13 smithi083 ceph-mon[29330]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:54:14.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:13 smithi094 ceph-mon[33032]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:54:14.113 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-25T11:54:14.113 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:54:14.114 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 607, 2024-04-25T11:54:14.114 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 783, 2024-04-25T11:54:14.114 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 1733374, 2024-04-25T11:54:14.114 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 268551, 2024-04-25T11:54:14.114 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 9, 2024-04-25T11:54:14.114 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 2, 2024-04-25T11:54:14.114 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.099665960, 2024-04-25T11:54:14.115 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.036175752, 2024-04-25T11:54:14.115 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.061223702, 2024-04-25T11:54:14.115 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.010185072, 2024-04-25T11:54:14.115 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:14.115 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 785, 2024-04-25T11:54:14.115 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.057700145, 2024-04-25T11:54:14.115 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000073503 2024-04-25T11:54:14.115 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.116 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:14.116 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 806, 2024-04-25T11:54:14.116 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000274170, 2024-04-25T11:54:14.116 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000340 2024-04-25T11:54:14.116 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.116 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1630160, 2024-04-25T11:54:14.116 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 135456 2024-04-25T11:54:14.116 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.117 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:54:14.117 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 23, 2024-04-25T11:54:14.117 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 185, 2024-04-25T11:54:14.117 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 5257, 2024-04-25T11:54:14.117 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 499666, 2024-04-25T11:54:14.117 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 17, 2024-04-25T11:54:14.117 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 6, 2024-04-25T11:54:14.117 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.014620316, 2024-04-25T11:54:14.118 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.007097164, 2024-04-25T11:54:14.118 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.005060709, 2024-04-25T11:54:14.118 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000215635, 2024-04-25T11:54:14.118 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:14.118 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 180, 2024-04-25T11:54:14.118 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.021202858, 2024-04-25T11:54:14.118 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000117793 2024-04-25T11:54:14.118 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.119 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:14.119 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 21, 2024-04-25T11:54:14.119 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000003479, 2024-04-25T11:54:14.119 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000165 2024-04-25T11:54:14.119 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.119 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 2176, 2024-04-25T11:54:14.119 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 442336 2024-04-25T11:54:14.119 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.120 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:54:14.120 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 117, 2024-04-25T11:54:14.120 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 119, 2024-04-25T11:54:14.120 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 46087, 2024-04-25T11:54:14.120 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 49929, 2024-04-25T11:54:14.120 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 8, 2024-04-25T11:54:14.120 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 6, 2024-04-25T11:54:14.120 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.011868740, 2024-04-25T11:54:14.121 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.004191991, 2024-04-25T11:54:14.121 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.008133243, 2024-04-25T11:54:14.121 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000475769, 2024-04-25T11:54:14.121 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:14.121 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 117, 2024-04-25T11:54:14.121 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.028074107, 2024-04-25T11:54:14.121 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000239949 2024-04-25T11:54:14.121 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.121 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:14.122 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 168, 2024-04-25T11:54:14.122 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000040324, 2024-04-25T11:54:14.122 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000240 2024-04-25T11:54:14.122 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.122 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:54:14.122 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:54:14.122 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.123 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-04-25T11:54:14.123 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-04-25T11:54:14.123 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-04-25T11:54:14.123 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.123 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-04-25T11:54:14.123 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-25T11:54:14.123 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-25T11:54:14.123 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 47, 2024-04-25T11:54:14.124 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.002637575, 2024-04-25T11:54:14.124 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000056118 2024-04-25T11:54:14.124 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:14.124 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.124 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-04-25T11:54:14.124 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-25T11:54:14.124 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-25T11:54:14.124 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-04-25T11:54:14.125 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000009371, 2024-04-25T11:54:14.125 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000004685 2024-04-25T11:54:14.125 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:14.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.125 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-04-25T11:54:14.125 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 0, 2024-04-25T11:54:14.125 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 0, 2024-04-25T11:54:14.125 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-04-25T11:54:14.125 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.126 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.126 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.126 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.126 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-04-25T11:54:14.126 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2024-04-25T11:54:14.126 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-04-25T11:54:14.126 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-04-25T11:54:14.126 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 11, 2024-04-25T11:54:14.127 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-04-25T11:54:14.127 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-04-25T11:54:14.127 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 10, 2024-04-25T11:54:14.127 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 0, 2024-04-25T11:54:14.127 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 10, 2024-04-25T11:54:14.127 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-04-25T11:54:14.127 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 10, 2024-04-25T11:54:14.127 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-04-25T11:54:14.127 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2024-04-25T11:54:14.128 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2024-04-25T11:54:14.128 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 1, 2024-04-25T11:54:14.128 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 0, 2024-04-25T11:54:14.128 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 0, 2024-04-25T11:54:14.128 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-04-25T11:54:14.128 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 0, 2024-04-25T11:54:14.128 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:54:14.128 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-04-25T11:54:14.129 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-04-25T11:54:14.129 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-04-25T11:54:14.129 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-04-25T11:54:14.129 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-04-25T11:54:14.129 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-04-25T11:54:14.129 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-04-25T11:54:14.129 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-04-25T11:54:14.129 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-04-25T11:54:14.130 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:54:14.130 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-04-25T11:54:14.130 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 3, 2024-04-25T11:54:14.130 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 117, 2024-04-25T11:54:14.130 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2024-04-25T11:54:14.130 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:54:14.130 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-04-25T11:54:14.130 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:54:14.131 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:54:14.131 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:54:14.131 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:54:14.131 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:54:14.131 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-04-25T11:54:14.131 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:54:14.131 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:54:14.131 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:54:14.132 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:54:14.132 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:54:14.132 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:54:14.132 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2024-04-25T11:54:14.132 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:54:14.132 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-04-25T11:54:14.132 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2024-04-25T11:54:14.132 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.132 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-04-25T11:54:14.133 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-04-25T11:54:14.133 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-04-25T11:54:14.133 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-04-25T11:54:14.133 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 1, 2024-04-25T11:54:14.133 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-04-25T11:54:14.133 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-04-25T11:54:14.133 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-04-25T11:54:14.133 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:54:14.134 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-04-25T11:54:14.134 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-04-25T11:54:14.134 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-04-25T11:54:14.134 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-04-25T11:54:14.134 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-04-25T11:54:14.134 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:54:14.134 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:54:14.134 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-04-25T11:54:14.135 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-04-25T11:54:14.135 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:54:14.135 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:54:14.135 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:54:14.135 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-04-25T11:54:14.135 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-04-25T11:54:14.135 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:54:14.135 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-04-25T11:54:14.135 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-04-25T11:54:14.136 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.136 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-04-25T11:54:14.136 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-04-25T11:54:14.136 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 6, 2024-04-25T11:54:14.136 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-04-25T11:54:14.136 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-04-25T11:54:14.136 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 6, 2024-04-25T11:54:14.136 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-04-25T11:54:14.136 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-04-25T11:54:14.137 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 2, 2024-04-25T11:54:14.137 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-04-25T11:54:14.137 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-04-25T11:54:14.137 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 2, 2024-04-25T11:54:14.137 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 2, 2024-04-25T11:54:14.137 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-04-25T11:54:14.137 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-04-25T11:54:14.137 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4194304, 2024-04-25T11:54:14.138 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 4195956, 2024-04-25T11:54:14.138 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 0, 2024-04-25T11:54:14.138 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-04-25T11:54:14.138 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 5, 2024-04-25T11:54:14.138 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.008912347, 2024-04-25T11:54:14.138 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001782469 2024-04-25T11:54:14.138 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.138 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 0 2024-04-25T11:54:14.138 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.139 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-04-25T11:54:14.139 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 13, 2024-04-25T11:54:14.139 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 13, 2024-04-25T11:54:14.139 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 0, 2024-04-25T11:54:14.139 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 11, 2024-04-25T11:54:14.139 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 12, 2024-04-25T11:54:14.139 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 1, 2024-04-25T11:54:14.139 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 10, 2024-04-25T11:54:14.139 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 10, 2024-04-25T11:54:14.140 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 0, 2024-04-25T11:54:14.140 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2024-04-25T11:54:14.140 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 0, 2024-04-25T11:54:14.140 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 0, 2024-04-25T11:54:14.140 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 51820, 2024-04-25T11:54:14.140 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 223608 2024-04-25T11:54:14.140 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.140 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-04-25T11:54:14.141 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 0, 2024-04-25T11:54:14.141 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-04-25T11:54:14.141 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 0, 2024-04-25T11:54:14.141 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 8, 2024-04-25T11:54:14.141 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 0, 2024-04-25T11:54:14.141 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-04-25T11:54:14.141 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.141 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.141 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.142 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.142 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-04-25T11:54:14.142 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.142 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.142 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.142 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.142 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-04-25T11:54:14.142 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.143 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.143 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.143 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.143 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-04-25T11:54:14.143 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.143 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.143 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.143 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.143 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-04-25T11:54:14.144 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.144 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.144 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.144 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.144 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-04-25T11:54:14.144 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.144 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.144 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.145 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.145 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-04-25T11:54:14.145 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.145 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.145 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.145 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.145 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-04-25T11:54:14.145 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.146 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.146 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.146 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.146 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-04-25T11:54:14.146 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.146 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.146 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.146 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.147 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-04-25T11:54:14.147 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.147 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.147 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.147 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.147 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-04-25T11:54:14.147 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.147 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.147 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.148 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.148 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-04-25T11:54:14.148 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.149 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.149 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.149 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.149 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-04-25T11:54:14.149 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.149 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.149 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.150 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.150 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-04-25T11:54:14.150 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.150 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.150 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.150 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.150 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-04-25T11:54:14.150 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.151 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.151 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.151 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.151 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-04-25T11:54:14.151 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.151 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.151 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.151 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.151 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-04-25T11:54:14.152 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.152 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.152 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.152 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.152 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-04-25T11:54:14.152 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.152 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.152 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.152 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.153 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-04-25T11:54:14.153 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.153 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.153 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.153 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.153 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-04-25T11:54:14.153 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.153 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.154 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.154 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.154 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-04-25T11:54:14.154 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.154 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.154 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.154 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.154 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-04-25T11:54:14.154 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.155 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.155 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.155 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.155 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-04-25T11:54:14.155 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.155 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.155 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.155 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.156 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-04-25T11:54:14.156 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.156 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.156 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.156 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.156 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-04-25T11:54:14.156 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.156 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.157 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.157 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.157 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-04-25T11:54:14.157 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.157 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.157 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.157 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.157 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-04-25T11:54:14.158 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.158 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.158 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.158 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.158 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-04-25T11:54:14.158 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.158 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.158 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.158 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.159 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-04-25T11:54:14.159 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.159 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.159 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.159 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.159 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:54:14.159 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:54:14.159 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-04-25T11:54:14.160 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.160 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.160 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.160 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:14.160 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.160 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-04-25T11:54:14.160 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 0, 2024-04-25T11:54:14.160 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 2, 2024-04-25T11:54:14.160 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 2, 2024-04-25T11:54:14.161 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 0, 2024-04-25T11:54:14.161 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-04-25T11:54:14.161 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 0, 2024-04-25T11:54:14.161 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 0, 2024-04-25T11:54:14.161 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 87, 2024-04-25T11:54:14.161 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-04-25T11:54:14.161 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.161 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-04-25T11:54:14.162 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:54:14.162 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-04-25T11:54:14.162 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:54:14.162 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:54:14.162 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:54:14.162 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:54:14.162 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:54:14.162 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:54:14.163 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:54:14.163 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:54:14.163 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:54:14.163 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:54:14.163 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:54:14.163 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:54:14.163 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:54:14.163 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-04-25T11:54:14.163 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:54:14.164 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-04-25T11:54:14.164 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:54:14.164 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:54:14.164 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:54:14.164 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:54:14.164 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:54:14.164 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:54:14.164 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:54:14.165 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-04-25T11:54:14.165 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:54:14.165 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:54:14.165 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:54:14.165 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-04-25T11:54:14.165 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-04-25T11:54:14.165 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-04-25T11:54:14.165 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:54:14.166 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:54:14.166 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:54:14.166 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:54:14.166 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 262375, 2024-04-25T11:54:14.166 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 86, 2024-04-25T11:54:14.166 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:54:14.166 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-04-25T11:54:14.166 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-04-25T11:54:14.167 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-04-25T11:54:14.167 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:54:14.167 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-04-25T11:54:14.167 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:54:14.167 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-04-25T11:54:14.167 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 8152, 2024-04-25T11:54:14.167 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 170, 2024-04-25T11:54:14.167 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:54:14.167 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:54:14.168 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-04-25T11:54:14.168 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-04-25T11:54:14.168 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 50872, 2024-04-25T11:54:14.168 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 198, 2024-04-25T11:54:14.168 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-04-25T11:54:14.168 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-04-25T11:54:14.168 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-04-25T11:54:14.168 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-04-25T11:54:14.169 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.169 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-04-25T11:54:14.169 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-04-25T11:54:14.169 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-04-25T11:54:14.169 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 41, 2024-04-25T11:54:14.169 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 11954, 2024-04-25T11:54:14.169 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 33, 2024-04-25T11:54:14.169 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 41, 2024-04-25T11:54:14.169 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-04-25T11:54:14.170 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 41, 2024-04-25T11:54:14.170 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.202996203, 2024-04-25T11:54:14.170 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.004951126 2024-04-25T11:54:14.170 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.170 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-04-25T11:54:14.170 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-04-25T11:54:14.170 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 41, 2024-04-25T11:54:14.170 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 63 2024-04-25T11:54:14.171 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.171 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 41, 2024-04-25T11:54:14.171 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 0, 2024-04-25T11:54:14.171 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 41, 2024-04-25T11:54:14.171 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-04-25T11:54:14.171 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-04-25T11:54:14.171 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 0, 2024-04-25T11:54:14.171 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 10, 2024-04-25T11:54:14.172 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 0, 2024-04-25T11:54:14.172 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 5, 2024-04-25T11:54:14.172 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 7, 2024-04-25T11:54:14.172 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-04-25T11:54:14.172 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-04-25T11:54:14.172 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-04-25T11:54:14.172 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-04-25T11:54:14.172 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 6, 2024-04-25T11:54:14.172 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-04-25T11:54:14.173 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-04-25T11:54:14.173 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-04-25T11:54:14.173 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 0, 2024-04-25T11:54:14.173 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 20, 2024-04-25T11:54:14.173 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:54:14.173 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-04-25T11:54:14.173 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:54:14.173 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-04-25T11:54:14.174 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-04-25T11:54:14.174 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-04-25T11:54:14.174 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:54:14.174 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-04-25T11:54:14.174 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:54:14.174 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-04-25T11:54:14.174 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-04-25T11:54:14.174 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-04-25T11:54:14.174 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-04-25T11:54:14.175 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-04-25T11:54:14.175 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-04-25T11:54:14.175 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-04-25T11:54:14.175 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-04-25T11:54:14.175 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-04-25T11:54:14.175 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-04-25T11:54:14.175 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-04-25T11:54:14.175 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-04-25T11:54:14.176 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-04-25T11:54:14.176 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-04-25T11:54:14.176 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-04-25T11:54:14.176 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-04-25T11:54:14.176 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-04-25T11:54:14.176 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 98, 2024-04-25T11:54:14.176 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-04-25T11:54:14.176 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 12, 2024-04-25T11:54:14.177 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 9, 2024-04-25T11:54:14.177 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 9, 2024-04-25T11:54:14.177 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-04-25T11:54:14.177 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-04-25T11:54:14.177 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 14, 2024-04-25T11:54:14.177 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 0, 2024-04-25T11:54:14.177 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 1 2024-04-25T11:54:14.177 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.177 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-04-25T11:54:14.178 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 1, 2024-04-25T11:54:14.178 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:54:14.178 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-04-25T11:54:14.178 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-04-25T11:54:14.178 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.178 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-04-25T11:54:14.178 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2024-04-25T11:54:14.178 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.179 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-04-25T11:54:14.179 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-04-25T11:54:14.179 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:54:14.179 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-04-25T11:54:14.179 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-04-25T11:54:14.179 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-04-25T11:54:14.179 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-04-25T11:54:14.179 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-04-25T11:54:14.179 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.180 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:54:14.180 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:14.180 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-25T11:54:14.180 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:14.180 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 743, 2024-04-25T11:54:14.180 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1701052, 2024-04-25T11:54:14.180 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:14.181 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 743, 2024-04-25T11:54:14.181 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-25T11:54:14.181 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-25T11:54:14.181 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 743, 2024-04-25T11:54:14.181 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1701052, 2024-04-25T11:54:14.181 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:14.181 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.181 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.182 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.182 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:14.182 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.182 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-04-25T11:54:14.182 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:14.182 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-25T11:54:14.182 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:14.182 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-25T11:54:14.183 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-25T11:54:14.183 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:14.183 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:14.183 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 41, 2024-04-25T11:54:14.183 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 11954, 2024-04-25T11:54:14.183 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 41, 2024-04-25T11:54:14.183 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 11954, 2024-04-25T11:54:14.183 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:14.183 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.184 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.184 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.184 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:14.184 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.184 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-04-25T11:54:14.184 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:14.184 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-04-25T11:54:14.184 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:14.185 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-25T11:54:14.185 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-25T11:54:14.185 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:14.185 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:14.185 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 41, 2024-04-25T11:54:14.185 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 41, 2024-04-25T11:54:14.185 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 41, 2024-04-25T11:54:14.185 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 41, 2024-04-25T11:54:14.186 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:14.186 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.186 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.186 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.186 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:14.186 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.186 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:54:14.186 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:14.187 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-25T11:54:14.187 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:14.187 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-25T11:54:14.187 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-25T11:54:14.187 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:14.187 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:14.187 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-25T11:54:14.187 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-25T11:54:14.188 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-04-25T11:54:14.188 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-04-25T11:54:14.188 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:14.188 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.188 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.188 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.188 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:14.188 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:14.189 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x55b7e85dc500": { 2024-04-25T11:54:14.189 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:14.189 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-25T11:54:14.189 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:14.189 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 6, 2024-04-25T11:54:14.189 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1652, 2024-04-25T11:54:14.189 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:14.189 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 6, 2024-04-25T11:54:14.189 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-25T11:54:14.190 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-25T11:54:14.190 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 5, 2024-04-25T11:54:14.190 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1652, 2024-04-25T11:54:14.190 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:14.190 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:14.190 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:14.190 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:14.190 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:14.191 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:14.191 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-25T11:54:14.879 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T11:54:14.880 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T11:54:14.880 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-25T11:54:14.880 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-25T11:54:14.880 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T11:54:14.880 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-25T11:54:14.880 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T11:54:14.880 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T11:54:15.118 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:54:15.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:15 smithi111 ceph-mon[32784]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:54:16.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:15 smithi094 ceph-mon[33032]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:54:16.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:15 smithi083 ceph-mon[29330]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:54:17.000 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-25T11:54:17.000 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:54:17.000 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 361, 2024-04-25T11:54:17.001 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 566, 2024-04-25T11:54:17.001 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 113491, 2024-04-25T11:54:17.001 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 147031, 2024-04-25T11:54:17.001 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 10, 2024-04-25T11:54:17.001 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 3, 2024-04-25T11:54:17.001 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.058175700, 2024-04-25T11:54:17.002 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.022195959, 2024-04-25T11:54:17.002 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.027402815, 2024-04-25T11:54:17.002 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.005362056, 2024-04-25T11:54:17.002 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:17.002 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 564, 2024-04-25T11:54:17.002 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.043941030, 2024-04-25T11:54:17.002 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000077909 2024-04-25T11:54:17.002 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.003 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:17.003 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 861, 2024-04-25T11:54:17.003 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000278181, 2024-04-25T11:54:17.003 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000323 2024-04-25T11:54:17.003 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.003 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T11:54:17.003 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T11:54:17.003 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.004 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:54:17.004 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 72, 2024-04-25T11:54:17.004 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 230, 2024-04-25T11:54:17.004 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 26057, 2024-04-25T11:54:17.004 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 494669, 2024-04-25T11:54:17.004 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 14, 2024-04-25T11:54:17.004 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 6, 2024-04-25T11:54:17.004 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.021518972, 2024-04-25T11:54:17.004 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.009019855, 2024-04-25T11:54:17.005 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.007940154, 2024-04-25T11:54:17.005 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000399020, 2024-04-25T11:54:17.005 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:17.005 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 227, 2024-04-25T11:54:17.005 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.027913127, 2024-04-25T11:54:17.005 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000122965 2024-04-25T11:54:17.005 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.005 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:17.006 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 104, 2024-04-25T11:54:17.006 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000023157, 2024-04-25T11:54:17.006 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000222 2024-04-25T11:54:17.006 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.006 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 2176, 2024-04-25T11:54:17.006 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 445088 2024-04-25T11:54:17.006 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.006 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:54:17.007 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 334, 2024-04-25T11:54:17.007 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 313, 2024-04-25T11:54:17.007 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 1668768, 2024-04-25T11:54:17.007 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 188138, 2024-04-25T11:54:17.007 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 9, 2024-04-25T11:54:17.007 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 4, 2024-04-25T11:54:17.007 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.053368189, 2024-04-25T11:54:17.007 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.016664465, 2024-04-25T11:54:17.007 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.040739572, 2024-04-25T11:54:17.008 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.005959517, 2024-04-25T11:54:17.008 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:17.008 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 313, 2024-04-25T11:54:17.008 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.026619087, 2024-04-25T11:54:17.008 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000085045 2024-04-25T11:54:17.008 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.008 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:17.008 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 101, 2024-04-25T11:54:17.009 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000068811, 2024-04-25T11:54:17.009 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000681 2024-04-25T11:54:17.009 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.009 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1645456, 2024-04-25T11:54:17.009 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 134512 2024-04-25T11:54:17.009 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.009 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-04-25T11:54:17.009 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-04-25T11:54:17.009 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-04-25T11:54:17.010 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.010 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-04-25T11:54:17.010 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-25T11:54:17.010 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-25T11:54:17.010 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 52, 2024-04-25T11:54:17.010 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.001697565, 2024-04-25T11:54:17.010 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000032645 2024-04-25T11:54:17.010 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:17.011 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.011 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-04-25T11:54:17.011 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-25T11:54:17.011 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-25T11:54:17.011 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-04-25T11:54:17.011 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000014431, 2024-04-25T11:54:17.011 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000007215 2024-04-25T11:54:17.011 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:17.011 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.012 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-04-25T11:54:17.012 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 0, 2024-04-25T11:54:17.012 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 0, 2024-04-25T11:54:17.012 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-04-25T11:54:17.012 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.012 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.012 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.012 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.013 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-04-25T11:54:17.013 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2024-04-25T11:54:17.013 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-04-25T11:54:17.013 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-04-25T11:54:17.013 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 11, 2024-04-25T11:54:17.013 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-04-25T11:54:17.013 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-04-25T11:54:17.013 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 10, 2024-04-25T11:54:17.013 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 0, 2024-04-25T11:54:17.014 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 10, 2024-04-25T11:54:17.014 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-04-25T11:54:17.014 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 10, 2024-04-25T11:54:17.014 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-04-25T11:54:17.014 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2024-04-25T11:54:17.014 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2024-04-25T11:54:17.014 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 1, 2024-04-25T11:54:17.015 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 0, 2024-04-25T11:54:17.015 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 0, 2024-04-25T11:54:17.015 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-04-25T11:54:17.015 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 0, 2024-04-25T11:54:17.015 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:54:17.015 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-04-25T11:54:17.015 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-04-25T11:54:17.015 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-04-25T11:54:17.015 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-04-25T11:54:17.016 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-04-25T11:54:17.016 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-04-25T11:54:17.016 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-04-25T11:54:17.016 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-04-25T11:54:17.016 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-04-25T11:54:17.016 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:54:17.016 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-04-25T11:54:17.016 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 3, 2024-04-25T11:54:17.017 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 117, 2024-04-25T11:54:17.017 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2024-04-25T11:54:17.017 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:54:17.017 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-04-25T11:54:17.017 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:54:17.017 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:54:17.017 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:54:17.017 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:54:17.017 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:54:17.018 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-04-25T11:54:17.018 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:54:17.018 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:54:17.018 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:54:17.018 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:54:17.018 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:54:17.018 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:54:17.018 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2024-04-25T11:54:17.019 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:54:17.019 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-04-25T11:54:17.019 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2024-04-25T11:54:17.019 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.019 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-04-25T11:54:17.019 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-04-25T11:54:17.019 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-04-25T11:54:17.019 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-04-25T11:54:17.019 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 1, 2024-04-25T11:54:17.020 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-04-25T11:54:17.020 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-04-25T11:54:17.020 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-04-25T11:54:17.020 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:54:17.020 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-04-25T11:54:17.020 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-04-25T11:54:17.020 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-04-25T11:54:17.020 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-04-25T11:54:17.021 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-04-25T11:54:17.021 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:54:17.021 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:54:17.021 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-04-25T11:54:17.021 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-04-25T11:54:17.021 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:54:17.021 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:54:17.021 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:54:17.022 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-04-25T11:54:17.022 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-04-25T11:54:17.022 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:54:17.022 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-04-25T11:54:17.022 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-04-25T11:54:17.022 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.023 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-04-25T11:54:17.023 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-04-25T11:54:17.023 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 6, 2024-04-25T11:54:17.023 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-04-25T11:54:17.023 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-04-25T11:54:17.023 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 6, 2024-04-25T11:54:17.023 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-04-25T11:54:17.023 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-04-25T11:54:17.024 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 2, 2024-04-25T11:54:17.024 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-04-25T11:54:17.024 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-04-25T11:54:17.024 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 2, 2024-04-25T11:54:17.024 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 2, 2024-04-25T11:54:17.024 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-04-25T11:54:17.024 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-04-25T11:54:17.024 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4194304, 2024-04-25T11:54:17.024 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 4195956, 2024-04-25T11:54:17.025 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 0, 2024-04-25T11:54:17.025 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-04-25T11:54:17.025 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 5, 2024-04-25T11:54:17.025 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.008275626, 2024-04-25T11:54:17.025 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001655125 2024-04-25T11:54:17.025 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.025 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 0 2024-04-25T11:54:17.025 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.026 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-04-25T11:54:17.026 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 13, 2024-04-25T11:54:17.026 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 13, 2024-04-25T11:54:17.026 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 0, 2024-04-25T11:54:17.026 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 11, 2024-04-25T11:54:17.026 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 12, 2024-04-25T11:54:17.026 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 1, 2024-04-25T11:54:17.026 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 10, 2024-04-25T11:54:17.027 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 10, 2024-04-25T11:54:17.027 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 0, 2024-04-25T11:54:17.027 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2024-04-25T11:54:17.027 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 0, 2024-04-25T11:54:17.027 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 0, 2024-04-25T11:54:17.027 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 52444, 2024-04-25T11:54:17.027 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 223608 2024-04-25T11:54:17.027 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.028 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-04-25T11:54:17.028 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 0, 2024-04-25T11:54:17.028 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-04-25T11:54:17.028 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 0, 2024-04-25T11:54:17.028 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 8, 2024-04-25T11:54:17.028 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 0, 2024-04-25T11:54:17.028 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-04-25T11:54:17.028 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.028 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.029 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.029 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.029 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-04-25T11:54:17.029 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.029 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.029 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.029 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.029 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-04-25T11:54:17.030 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.030 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.030 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.030 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.030 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-04-25T11:54:17.030 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.030 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.030 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.031 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.031 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-04-25T11:54:17.031 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.031 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.031 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.031 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.031 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-04-25T11:54:17.031 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.031 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.032 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.032 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.032 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-04-25T11:54:17.032 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.032 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.032 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.032 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.032 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-04-25T11:54:17.033 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.033 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.033 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.033 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.033 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-04-25T11:54:17.033 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.033 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.033 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.033 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.034 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-04-25T11:54:17.034 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.034 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.034 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.034 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.034 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-04-25T11:54:17.034 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.034 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.035 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.035 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.035 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-04-25T11:54:17.035 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.036 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.036 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.036 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.036 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-04-25T11:54:17.036 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.036 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.037 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.037 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.037 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-04-25T11:54:17.037 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.037 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.037 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.037 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.037 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-04-25T11:54:17.037 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.038 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.038 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.038 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.038 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-04-25T11:54:17.038 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.038 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.039 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.039 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.039 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-04-25T11:54:17.039 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.039 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.039 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.039 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.039 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-04-25T11:54:17.040 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.040 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.040 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.040 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.040 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-04-25T11:54:17.040 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.040 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.040 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.040 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.041 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-04-25T11:54:17.041 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.041 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.041 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.041 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.041 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-04-25T11:54:17.041 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.041 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.041 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.042 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.042 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-04-25T11:54:17.042 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.042 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.042 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.042 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.042 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-04-25T11:54:17.042 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.043 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.043 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.043 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.043 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-04-25T11:54:17.043 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.043 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.043 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.043 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.043 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-04-25T11:54:17.044 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.044 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.044 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.044 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.044 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-04-25T11:54:17.044 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.044 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.044 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.044 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.045 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-04-25T11:54:17.045 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.045 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.045 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.045 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.045 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-04-25T11:54:17.045 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.045 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.046 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.046 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.046 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-04-25T11:54:17.046 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.046 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.046 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.046 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.046 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:54:17.047 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:54:17.047 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-04-25T11:54:17.047 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.047 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.047 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.047 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:17.047 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.047 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-04-25T11:54:17.047 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 0, 2024-04-25T11:54:17.048 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 2, 2024-04-25T11:54:17.048 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 2, 2024-04-25T11:54:17.048 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 0, 2024-04-25T11:54:17.048 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-04-25T11:54:17.048 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 0, 2024-04-25T11:54:17.048 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 0, 2024-04-25T11:54:17.048 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 86, 2024-04-25T11:54:17.048 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-04-25T11:54:17.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.049 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-04-25T11:54:17.049 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:54:17.049 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-04-25T11:54:17.049 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:54:17.049 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:54:17.049 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:54:17.049 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:54:17.049 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:54:17.050 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:54:17.050 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:54:17.050 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:54:17.050 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:54:17.050 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:54:17.050 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:54:17.050 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:54:17.050 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:54:17.050 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-04-25T11:54:17.051 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:54:17.051 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-04-25T11:54:17.051 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:54:17.051 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:54:17.051 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:54:17.051 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:54:17.051 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:54:17.051 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:54:17.051 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:54:17.052 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-04-25T11:54:17.052 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:54:17.052 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:54:17.052 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:54:17.052 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-04-25T11:54:17.052 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-04-25T11:54:17.052 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-04-25T11:54:17.052 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:54:17.053 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:54:17.053 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:54:17.053 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:54:17.053 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 262295, 2024-04-25T11:54:17.053 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 86, 2024-04-25T11:54:17.053 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:54:17.053 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-04-25T11:54:17.053 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-04-25T11:54:17.053 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-04-25T11:54:17.054 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:54:17.054 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-04-25T11:54:17.054 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:54:17.054 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-04-25T11:54:17.054 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 8152, 2024-04-25T11:54:17.054 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 170, 2024-04-25T11:54:17.054 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:54:17.054 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:54:17.055 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-04-25T11:54:17.055 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-04-25T11:54:17.055 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 50872, 2024-04-25T11:54:17.055 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 198, 2024-04-25T11:54:17.055 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-04-25T11:54:17.055 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-04-25T11:54:17.055 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-04-25T11:54:17.055 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-04-25T11:54:17.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.056 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-04-25T11:54:17.056 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-04-25T11:54:17.056 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-04-25T11:54:17.056 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 42, 2024-04-25T11:54:17.056 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 12044, 2024-04-25T11:54:17.056 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 30, 2024-04-25T11:54:17.056 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 42, 2024-04-25T11:54:17.056 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-04-25T11:54:17.057 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 42, 2024-04-25T11:54:17.057 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.225995748, 2024-04-25T11:54:17.057 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.005380851 2024-04-25T11:54:17.057 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.057 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-04-25T11:54:17.057 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-04-25T11:54:17.057 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 42, 2024-04-25T11:54:17.057 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 64 2024-04-25T11:54:17.057 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.058 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 42, 2024-04-25T11:54:17.058 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 0, 2024-04-25T11:54:17.058 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 42, 2024-04-25T11:54:17.058 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-04-25T11:54:17.058 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-04-25T11:54:17.058 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 0, 2024-04-25T11:54:17.058 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 10, 2024-04-25T11:54:17.058 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 0, 2024-04-25T11:54:17.059 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 5, 2024-04-25T11:54:17.059 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 8, 2024-04-25T11:54:17.059 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-04-25T11:54:17.059 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-04-25T11:54:17.059 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-04-25T11:54:17.059 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-04-25T11:54:17.059 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 6, 2024-04-25T11:54:17.059 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-04-25T11:54:17.059 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-04-25T11:54:17.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-04-25T11:54:17.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 0, 2024-04-25T11:54:17.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 20, 2024-04-25T11:54:17.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:54:17.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-04-25T11:54:17.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:54:17.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-04-25T11:54:17.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-04-25T11:54:17.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-04-25T11:54:17.061 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:54:17.061 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-04-25T11:54:17.061 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:54:17.061 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-04-25T11:54:17.061 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-04-25T11:54:17.061 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-04-25T11:54:17.061 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-04-25T11:54:17.061 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-04-25T11:54:17.062 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-04-25T11:54:17.062 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-04-25T11:54:17.062 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-04-25T11:54:17.062 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-04-25T11:54:17.062 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-04-25T11:54:17.062 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-04-25T11:54:17.062 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-04-25T11:54:17.062 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-04-25T11:54:17.062 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-04-25T11:54:17.063 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-04-25T11:54:17.063 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-04-25T11:54:17.063 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-04-25T11:54:17.063 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 99, 2024-04-25T11:54:17.063 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-04-25T11:54:17.063 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 13, 2024-04-25T11:54:17.063 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 9, 2024-04-25T11:54:17.063 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 9, 2024-04-25T11:54:17.063 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-04-25T11:54:17.064 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-04-25T11:54:17.064 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 14, 2024-04-25T11:54:17.064 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 0, 2024-04-25T11:54:17.064 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 1 2024-04-25T11:54:17.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.064 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-04-25T11:54:17.064 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 1, 2024-04-25T11:54:17.064 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:54:17.065 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-04-25T11:54:17.065 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-04-25T11:54:17.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.065 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-04-25T11:54:17.065 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2024-04-25T11:54:17.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.065 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-04-25T11:54:17.065 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-04-25T11:54:17.065 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:54:17.066 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-04-25T11:54:17.066 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-04-25T11:54:17.066 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-04-25T11:54:17.066 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-04-25T11:54:17.066 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-04-25T11:54:17.066 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.066 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:54:17.066 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:17.066 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-25T11:54:17.067 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:17.067 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 763, 2024-04-25T11:54:17.067 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1722726, 2024-04-25T11:54:17.067 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:17.067 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 763, 2024-04-25T11:54:17.067 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-25T11:54:17.067 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-25T11:54:17.067 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 763, 2024-04-25T11:54:17.068 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1722726, 2024-04-25T11:54:17.068 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:17.068 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.068 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.068 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.068 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:17.068 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.068 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-04-25T11:54:17.068 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:17.069 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-25T11:54:17.069 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:17.069 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-25T11:54:17.069 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-25T11:54:17.069 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:17.069 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:17.069 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 42, 2024-04-25T11:54:17.069 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 12044, 2024-04-25T11:54:17.070 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 42, 2024-04-25T11:54:17.070 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 12044, 2024-04-25T11:54:17.070 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:17.070 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.070 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.070 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.070 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:17.070 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.070 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-04-25T11:54:17.071 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:17.071 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-04-25T11:54:17.071 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:17.071 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-25T11:54:17.071 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-25T11:54:17.071 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:17.071 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:17.071 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 42, 2024-04-25T11:54:17.072 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 42, 2024-04-25T11:54:17.072 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 42, 2024-04-25T11:54:17.072 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 42, 2024-04-25T11:54:17.072 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:17.072 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.072 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.072 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.072 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:17.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.073 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:54:17.073 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:17.073 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-25T11:54:17.073 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:17.073 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-25T11:54:17.073 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-25T11:54:17.073 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:17.073 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:17.073 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-25T11:54:17.074 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-25T11:54:17.074 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-04-25T11:54:17.074 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-04-25T11:54:17.074 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:17.074 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.074 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.074 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.074 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:17.075 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:17.075 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x55b058b72500": { 2024-04-25T11:54:17.075 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:17.075 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-25T11:54:17.075 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:17.075 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 6, 2024-04-25T11:54:17.075 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1652, 2024-04-25T11:54:17.075 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:17.075 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 6, 2024-04-25T11:54:17.076 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-25T11:54:17.076 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-25T11:54:17.076 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 5, 2024-04-25T11:54:17.076 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1652, 2024-04-25T11:54:17.076 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:17.076 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:17.076 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:17.076 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:17.077 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:17.077 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:17.077 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-25T11:54:17.794 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T11:54:17.794 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T11:54:17.794 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=0 2024-04-25T11:54:17.795 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T11:54:17.795 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T11:54:17.795 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-25T11:54:17.795 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T11:54:17.795 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T11:54:18.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:17 smithi094 ceph-mon[33032]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:54:18.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:17 smithi094 ceph-mon[33032]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T11:54:18.031 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T11:54:18.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:17 smithi111 ceph-mon[32784]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:54:18.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:17 smithi111 ceph-mon[32784]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T11:54:18.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:17 smithi083 ceph-mon[29330]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-25T11:54:18.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:17 smithi083 ceph-mon[29330]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T11:54:19.653 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-25T11:54:19.654 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T11:54:19.654 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 759, 2024-04-25T11:54:19.654 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 759, 2024-04-25T11:54:19.654 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 203888, 2024-04-25T11:54:19.654 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 240602, 2024-04-25T11:54:19.654 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 6, 2024-04-25T11:54:19.655 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 2, 2024-04-25T11:54:19.655 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.065486994, 2024-04-25T11:54:19.655 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.022424975, 2024-04-25T11:54:19.655 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.016755663, 2024-04-25T11:54:19.655 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.019077778, 2024-04-25T11:54:19.655 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:19.655 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 758, 2024-04-25T11:54:19.655 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.028127492, 2024-04-25T11:54:19.656 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000037107 2024-04-25T11:54:19.656 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.656 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:19.656 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.656 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.656 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.656 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.656 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T11:54:19.657 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T11:54:19.657 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.657 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T11:54:19.657 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 902, 2024-04-25T11:54:19.657 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 890, 2024-04-25T11:54:19.657 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 1782798, 2024-04-25T11:54:19.657 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 343469, 2024-04-25T11:54:19.658 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-04-25T11:54:19.658 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 3, 2024-04-25T11:54:19.658 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.108409297, 2024-04-25T11:54:19.658 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.032632277, 2024-04-25T11:54:19.658 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.051543307, 2024-04-25T11:54:19.658 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.028709928, 2024-04-25T11:54:19.658 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:19.659 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 889, 2024-04-25T11:54:19.659 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.032445203, 2024-04-25T11:54:19.659 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000036496 2024-04-25T11:54:19.659 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.659 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:19.659 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.659 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.659 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.660 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.660 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1620416, 2024-04-25T11:54:19.660 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 133584 2024-04-25T11:54:19.660 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.660 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T11:54:19.660 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 749, 2024-04-25T11:54:19.660 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 909, 2024-04-25T11:54:19.660 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 251312, 2024-04-25T11:54:19.661 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 679770, 2024-04-25T11:54:19.661 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 6, 2024-04-25T11:54:19.661 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 4, 2024-04-25T11:54:19.661 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.103637825, 2024-04-25T11:54:19.661 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.034125983, 2024-04-25T11:54:19.661 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.025588365, 2024-04-25T11:54:19.661 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.032994639, 2024-04-25T11:54:19.662 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T11:54:19.662 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 906, 2024-04-25T11:54:19.662 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.058414583, 2024-04-25T11:54:19.662 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000064475 2024-04-25T11:54:19.662 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.662 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-25T11:54:19.662 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.663 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.663 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.663 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.663 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T11:54:19.663 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 442224 2024-04-25T11:54:19.663 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.663 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-04-25T11:54:19.663 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-04-25T11:54:19.664 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-04-25T11:54:19.664 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.664 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-04-25T11:54:19.664 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-25T11:54:19.664 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-25T11:54:19.664 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3576, 2024-04-25T11:54:19.664 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.116991688, 2024-04-25T11:54:19.664 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000032715 2024-04-25T11:54:19.665 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:19.665 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.665 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-04-25T11:54:19.665 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-25T11:54:19.665 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-25T11:54:19.665 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.665 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.665 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.666 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:19.666 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.666 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-04-25T11:54:19.666 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 0, 2024-04-25T11:54:19.666 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 0, 2024-04-25T11:54:19.666 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-04-25T11:54:19.666 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.667 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.667 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.667 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.667 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-04-25T11:54:19.667 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2024-04-25T11:54:19.667 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-04-25T11:54:19.667 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-04-25T11:54:19.667 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 0, 2024-04-25T11:54:19.668 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-04-25T11:54:19.668 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-04-25T11:54:19.668 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 0, 2024-04-25T11:54:19.668 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 0, 2024-04-25T11:54:19.668 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 0, 2024-04-25T11:54:19.668 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-04-25T11:54:19.668 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 0, 2024-04-25T11:54:19.668 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-04-25T11:54:19.669 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2024-04-25T11:54:19.669 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2024-04-25T11:54:19.669 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 1, 2024-04-25T11:54:19.669 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 0, 2024-04-25T11:54:19.669 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 0, 2024-04-25T11:54:19.669 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-04-25T11:54:19.669 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 0, 2024-04-25T11:54:19.669 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-04-25T11:54:19.670 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-04-25T11:54:19.670 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-04-25T11:54:19.671 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-04-25T11:54:19.671 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-04-25T11:54:19.671 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-04-25T11:54:19.671 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-04-25T11:54:19.671 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-04-25T11:54:19.671 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-04-25T11:54:19.671 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-04-25T11:54:19.671 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-04-25T11:54:19.672 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-04-25T11:54:19.672 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 0, 2024-04-25T11:54:19.672 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 0, 2024-04-25T11:54:19.672 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2024-04-25T11:54:19.672 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T11:54:19.673 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-04-25T11:54:19.673 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T11:54:19.673 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-04-25T11:54:19.673 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-04-25T11:54:19.673 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T11:54:19.673 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T11:54:19.673 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-04-25T11:54:19.673 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-04-25T11:54:19.674 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T11:54:19.674 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2024-04-25T11:54:19.674 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T11:54:19.674 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T11:54:19.674 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T11:54:19.674 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2024-04-25T11:54:19.674 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-04-25T11:54:19.674 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-04-25T11:54:19.675 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2024-04-25T11:54:19.675 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.675 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-04-25T11:54:19.675 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-04-25T11:54:19.675 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-04-25T11:54:19.675 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-04-25T11:54:19.675 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 0, 2024-04-25T11:54:19.675 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-04-25T11:54:19.675 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-04-25T11:54:19.676 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-04-25T11:54:19.676 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-04-25T11:54:19.676 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-04-25T11:54:19.676 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-04-25T11:54:19.676 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-04-25T11:54:19.676 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-04-25T11:54:19.676 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-04-25T11:54:19.676 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-04-25T11:54:19.677 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-04-25T11:54:19.677 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-04-25T11:54:19.677 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-04-25T11:54:19.677 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 0, 2024-04-25T11:54:19.677 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 0, 2024-04-25T11:54:19.677 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T11:54:19.677 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-04-25T11:54:19.677 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-04-25T11:54:19.677 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-04-25T11:54:19.678 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-04-25T11:54:19.678 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-04-25T11:54:19.678 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.678 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-04-25T11:54:19.678 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-04-25T11:54:19.678 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 0, 2024-04-25T11:54:19.678 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-04-25T11:54:19.678 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-04-25T11:54:19.679 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 9, 2024-04-25T11:54:19.679 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-04-25T11:54:19.679 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-04-25T11:54:19.679 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 0, 2024-04-25T11:54:19.679 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-04-25T11:54:19.679 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-04-25T11:54:19.682 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 2, 2024-04-25T11:54:19.682 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 2, 2024-04-25T11:54:19.682 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-04-25T11:54:19.682 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-04-25T11:54:19.682 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4194304, 2024-04-25T11:54:19.682 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 4196879, 2024-04-25T11:54:19.682 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 4196738, 2024-04-25T11:54:19.683 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-04-25T11:54:19.683 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.683 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.683 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.683 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.683 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 9 2024-04-25T11:54:19.683 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.683 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-04-25T11:54:19.684 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 3, 2024-04-25T11:54:19.684 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 3, 2024-04-25T11:54:19.684 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 0, 2024-04-25T11:54:19.684 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 1, 2024-04-25T11:54:19.684 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 1, 2024-04-25T11:54:19.684 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 0, 2024-04-25T11:54:19.684 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 0, 2024-04-25T11:54:19.684 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 0, 2024-04-25T11:54:19.684 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 0, 2024-04-25T11:54:19.685 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2024-04-25T11:54:19.685 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 0, 2024-04-25T11:54:19.685 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 0, 2024-04-25T11:54:19.685 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 49772, 2024-04-25T11:54:19.685 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 215416 2024-04-25T11:54:19.685 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.685 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-04-25T11:54:19.685 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 0, 2024-04-25T11:54:19.686 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-04-25T11:54:19.686 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 0, 2024-04-25T11:54:19.686 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 0, 2024-04-25T11:54:19.686 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 0, 2024-04-25T11:54:19.686 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-04-25T11:54:19.686 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.686 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.686 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.687 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.687 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-04-25T11:54:19.687 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.687 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.687 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.687 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.687 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-04-25T11:54:19.687 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.688 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.688 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.688 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.688 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-04-25T11:54:19.688 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.688 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.688 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.688 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.688 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-04-25T11:54:19.689 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.689 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.689 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.689 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.689 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-04-25T11:54:19.689 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.689 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.689 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.689 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.690 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-04-25T11:54:19.690 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.690 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.690 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.690 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.690 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-04-25T11:54:19.690 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.690 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.691 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.691 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.691 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-04-25T11:54:19.691 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.691 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.691 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.691 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.691 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-04-25T11:54:19.691 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.692 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.692 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.692 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.692 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-04-25T11:54:19.692 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.692 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.692 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.692 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.692 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-04-25T11:54:19.693 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.693 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.694 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.694 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.694 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-04-25T11:54:19.694 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.694 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.694 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.694 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.694 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-04-25T11:54:19.695 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.695 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.695 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.695 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.695 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-04-25T11:54:19.695 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.695 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.695 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.695 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.696 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-04-25T11:54:19.696 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.696 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.696 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.696 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.696 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-04-25T11:54:19.696 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.696 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.696 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.697 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.697 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-04-25T11:54:19.697 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.697 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.697 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.697 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.697 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-04-25T11:54:19.697 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.697 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.698 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.698 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.698 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-04-25T11:54:19.698 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.698 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.698 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.698 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.698 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-04-25T11:54:19.699 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.699 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.699 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.699 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.699 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-04-25T11:54:19.699 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.699 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.699 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.700 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.700 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-04-25T11:54:19.700 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.700 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.700 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.700 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.700 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-04-25T11:54:19.700 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.700 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.701 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.701 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.701 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-04-25T11:54:19.701 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.701 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.701 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.701 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.701 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-04-25T11:54:19.701 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.702 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.702 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.702 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.702 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-04-25T11:54:19.702 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.702 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.702 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.702 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.703 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-04-25T11:54:19.703 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.703 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.703 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.703 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.703 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-04-25T11:54:19.703 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.703 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.703 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.704 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.704 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-04-25T11:54:19.704 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-04-25T11:54:19.704 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-04-25T11:54:19.704 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.704 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.704 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.704 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:19.704 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.705 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-04-25T11:54:19.705 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 1, 2024-04-25T11:54:19.705 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 4, 2024-04-25T11:54:19.705 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 3, 2024-04-25T11:54:19.705 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 3, 2024-04-25T11:54:19.705 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-04-25T11:54:19.705 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 0, 2024-04-25T11:54:19.705 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 0, 2024-04-25T11:54:19.706 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 3981, 2024-04-25T11:54:19.706 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-04-25T11:54:19.706 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.706 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-04-25T11:54:19.706 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-04-25T11:54:19.706 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-04-25T11:54:19.706 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T11:54:19.706 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-04-25T11:54:19.706 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T11:54:19.707 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-04-25T11:54:19.707 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T11:54:19.707 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T11:54:19.707 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T11:54:19.707 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T11:54:19.707 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T11:54:19.707 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-04-25T11:54:19.707 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T11:54:19.707 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T11:54:19.708 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-04-25T11:54:19.708 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-04-25T11:54:19.708 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-04-25T11:54:19.708 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-04-25T11:54:19.708 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T11:54:19.708 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T11:54:19.708 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T11:54:19.708 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T11:54:19.709 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T11:54:19.709 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-04-25T11:54:19.709 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-04-25T11:54:19.709 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-04-25T11:54:19.709 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T11:54:19.709 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T11:54:19.709 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-04-25T11:54:19.709 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-04-25T11:54:19.709 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-04-25T11:54:19.710 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-04-25T11:54:19.710 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T11:54:19.710 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-04-25T11:54:19.710 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T11:54:19.710 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-04-25T11:54:19.710 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 189383, 2024-04-25T11:54:19.710 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 50, 2024-04-25T11:54:19.710 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-04-25T11:54:19.711 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-04-25T11:54:19.711 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-04-25T11:54:19.711 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-04-25T11:54:19.711 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-04-25T11:54:19.711 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-04-25T11:54:19.711 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-04-25T11:54:19.711 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-04-25T11:54:19.711 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 8032, 2024-04-25T11:54:19.711 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 167, 2024-04-25T11:54:19.712 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T11:54:19.712 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-04-25T11:54:19.712 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-04-25T11:54:19.712 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-04-25T11:54:19.712 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 8512, 2024-04-25T11:54:19.712 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 13, 2024-04-25T11:54:19.712 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-04-25T11:54:19.712 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-04-25T11:54:19.712 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-04-25T11:54:19.713 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-04-25T11:54:19.713 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.713 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-04-25T11:54:19.713 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-04-25T11:54:19.713 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-04-25T11:54:19.713 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 2162, 2024-04-25T11:54:19.713 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 21, 2024-04-25T11:54:19.713 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 3, 2024-04-25T11:54:19.714 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 2162, 2024-04-25T11:54:19.714 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-04-25T11:54:19.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2162, 2024-04-25T11:54:19.714 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 1.100980127, 2024-04-25T11:54:19.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000509241 2024-04-25T11:54:19.714 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.714 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-04-25T11:54:19.714 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-04-25T11:54:19.715 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2162, 2024-04-25T11:54:19.715 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 2163 2024-04-25T11:54:19.715 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.715 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 2162, 2024-04-25T11:54:19.715 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 2162, 2024-04-25T11:54:19.715 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 0, 2024-04-25T11:54:19.715 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-04-25T11:54:19.715 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-04-25T11:54:19.716 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 1405, 2024-04-25T11:54:19.716 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 0, 2024-04-25T11:54:19.716 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 755, 2024-04-25T11:54:19.716 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 0, 2024-04-25T11:54:19.716 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 0, 2024-04-25T11:54:19.716 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-04-25T11:54:19.716 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-04-25T11:54:19.717 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-04-25T11:54:19.717 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-04-25T11:54:19.717 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 0, 2024-04-25T11:54:19.717 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-04-25T11:54:19.717 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-04-25T11:54:19.717 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-04-25T11:54:19.717 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 1, 2024-04-25T11:54:19.717 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 0, 2024-04-25T11:54:19.718 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-04-25T11:54:19.718 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-04-25T11:54:19.718 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-04-25T11:54:19.718 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-04-25T11:54:19.718 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-04-25T11:54:19.718 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-04-25T11:54:19.718 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T11:54:19.718 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-04-25T11:54:19.719 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-04-25T11:54:19.719 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-04-25T11:54:19.719 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-04-25T11:54:19.719 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-04-25T11:54:19.719 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-04-25T11:54:19.719 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-04-25T11:54:19.719 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-04-25T11:54:19.719 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-04-25T11:54:19.719 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-04-25T11:54:19.720 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-04-25T11:54:19.720 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-04-25T11:54:19.720 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-04-25T11:54:19.720 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-04-25T11:54:19.720 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-04-25T11:54:19.720 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-04-25T11:54:19.720 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-04-25T11:54:19.720 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-04-25T11:54:19.721 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-04-25T11:54:19.721 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 99, 2024-04-25T11:54:19.721 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-04-25T11:54:19.721 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 13, 2024-04-25T11:54:19.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 5, 2024-04-25T11:54:19.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 5, 2024-04-25T11:54:19.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-04-25T11:54:19.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-04-25T11:54:19.721 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 0, 2024-04-25T11:54:19.722 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 2, 2024-04-25T11:54:19.722 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 0 2024-04-25T11:54:19.722 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.722 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-04-25T11:54:19.722 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 0, 2024-04-25T11:54:19.722 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-04-25T11:54:19.722 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-04-25T11:54:19.722 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-04-25T11:54:19.723 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.723 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-04-25T11:54:19.723 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2024-04-25T11:54:19.723 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.723 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-04-25T11:54:19.723 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-04-25T11:54:19.723 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T11:54:19.723 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-04-25T11:54:19.723 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-04-25T11:54:19.724 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-04-25T11:54:19.724 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-04-25T11:54:19.724 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-04-25T11:54:19.724 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.724 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T11:54:19.724 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:19.724 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-25T11:54:19.724 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:19.724 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 2406, 2024-04-25T11:54:19.725 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 2006134, 2024-04-25T11:54:19.725 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:19.725 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 2406, 2024-04-25T11:54:19.725 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-25T11:54:19.725 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-25T11:54:19.725 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2406, 2024-04-25T11:54:19.725 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 2006134, 2024-04-25T11:54:19.725 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:19.725 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.726 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.726 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.726 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:19.726 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.726 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-04-25T11:54:19.726 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:19.726 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-25T11:54:19.726 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:19.727 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-25T11:54:19.727 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-25T11:54:19.727 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:19.727 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:19.727 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 2162, 2024-04-25T11:54:19.727 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 2580, 2024-04-25T11:54:19.727 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2162, 2024-04-25T11:54:19.727 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 2580, 2024-04-25T11:54:19.728 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:19.728 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.728 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.728 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.728 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:19.728 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.728 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-04-25T11:54:19.728 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:19.728 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-04-25T11:54:19.729 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:19.729 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-25T11:54:19.729 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-25T11:54:19.729 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:19.729 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:19.729 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 2162, 2024-04-25T11:54:19.729 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 2162, 2024-04-25T11:54:19.729 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2162, 2024-04-25T11:54:19.730 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 2162, 2024-04-25T11:54:19.730 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:19.730 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.730 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.730 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.730 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:19.730 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.730 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-04-25T11:54:19.731 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:19.731 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-25T11:54:19.731 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:19.731 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-25T11:54:19.731 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-25T11:54:19.731 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:19.731 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:19.731 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-25T11:54:19.731 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-25T11:54:19.732 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-04-25T11:54:19.732 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-04-25T11:54:19.732 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:19.732 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.732 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.732 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.732 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:19.732 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-25T11:54:19.733 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x5643500466e0": { 2024-04-25T11:54:19.733 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-25T11:54:19.733 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-25T11:54:19.733 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-25T11:54:19.733 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-25T11:54:19.733 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-25T11:54:19.733 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-25T11:54:19.733 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-25T11:54:19.733 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-25T11:54:19.734 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-25T11:54:19.734 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-04-25T11:54:19.734 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-04-25T11:54:19.734 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-25T11:54:19.734 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-25T11:54:19.734 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-25T11:54:19.734 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-25T11:54:19.734 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:19.734 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-25T11:54:19.735 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-25T11:54:20.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:19 smithi094 ceph-mon[33032]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T11:54:20.075 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:19 smithi111 ceph-mon[32784]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T11:54:20.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:19 smithi083 ceph-mon[29330]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-25T11:54:20.562 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-25T11:54:20.562 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-25T11:54:20.562 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-04-25T11:54:20.563 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-25T11:54:20.563 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T11:54:20.563 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-04-25T11:54:20.563 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/github.com_ceph_ceph_df40303764f976ac2e973c1ef2b40d1ebbda473b/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.dir_split', 'mds.imported', 'mds.exported'} 2024-04-25T11:54:20.573 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T11:54:20.585 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T11:54:20.585 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:54:20.585 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:54:20.610 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:54:20.611 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:54:20.670 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:54:20.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.833+0000 7efc25d19640 1 -- 172.21.15.83:0/708652579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc20101650 msgr2=0x7efc2010e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:20.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.833+0000 7efc25d19640 1 --2- 172.21.15.83:0/708652579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc20101650 0x7efc2010e3a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efc14009900 tx=0x7efc1402a410 comp rx=0 tx=0).stop 2024-04-25T11:54:20.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.833+0000 7efc25d19640 1 -- 172.21.15.83:0/708652579 shutdown_connections 2024-04-25T11:54:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.833+0000 7efc25d19640 1 --2- 172.21.15.83:0/708652579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc20101650 0x7efc2010e3a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.833+0000 7efc25d19640 1 --2- 172.21.15.83:0/708652579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc2010e8e0 0x7efc20110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.833+0000 7efc25d19640 1 --2- 172.21.15.83:0/708652579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc20100d30 0x7efc20101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.833+0000 7efc25d19640 1 -- 172.21.15.83:0/708652579 >> 172.21.15.83:0/708652579 conn(0x7efc200771d0 msgr2=0x7efc200775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.833+0000 7efc25d19640 1 -- 172.21.15.83:0/708652579 shutdown_connections 2024-04-25T11:54:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.833+0000 7efc25d19640 1 -- 172.21.15.83:0/708652579 wait complete. 2024-04-25T11:54:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.834+0000 7efc25d19640 1 Processor -- start 2024-04-25T11:54:20.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.834+0000 7efc25d19640 1 -- start start 2024-04-25T11:54:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.834+0000 7efc25d19640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc20100d30 0x7efc201a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.834+0000 7efc25d19640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc20101650 0x7efc201a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.834+0000 7efc25d19640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2010e8e0 0x7efc2019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.834+0000 7efc25d19640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc201143e0 con 0x7efc2010e8e0 2024-04-25T11:54:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.834+0000 7efc25d19640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efc20114260 con 0x7efc20101650 2024-04-25T11:54:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.834+0000 7efc25d19640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efc20114560 con 0x7efc20100d30 2024-04-25T11:54:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.834+0000 7efc1ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2010e8e0 0x7efc2019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.834+0000 7efc1ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2010e8e0 0x7efc2019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51256/0 (socket says 172.21.15.83:51256) 2024-04-25T11:54:20.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.834+0000 7efc1ffff640 1 -- 172.21.15.83:0/980899778 learned_addr learned my addr 172.21.15.83:0/980899778 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:54:20.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.834+0000 7efc1f7fe640 1 --2- 172.21.15.83:0/980899778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc20100d30 0x7efc201a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:20.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.834+0000 7efc1effd640 1 --2- 172.21.15.83:0/980899778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc20101650 0x7efc201a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:20.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.835+0000 7efc1ffff640 1 -- 172.21.15.83:0/980899778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc20100d30 msgr2=0x7efc201a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:20.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.835+0000 7efc1ffff640 1 --2- 172.21.15.83:0/980899778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc20100d30 0x7efc201a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:20.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.835+0000 7efc1ffff640 1 -- 172.21.15.83:0/980899778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc20101650 msgr2=0x7efc201a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:20.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.835+0000 7efc1ffff640 1 --2- 172.21.15.83:0/980899778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc20101650 0x7efc201a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:20.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.835+0000 7efc1ffff640 1 -- 172.21.15.83:0/980899778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2019afc0 con 0x7efc2010e8e0 2024-04-25T11:54:20.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.835+0000 7efc1effd640 1 --2- 172.21.15.83:0/980899778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc20101650 0x7efc201a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:54:20.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.835+0000 7efc1ffff640 1 --2- 172.21.15.83:0/980899778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2010e8e0 0x7efc2019a700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efc1800d6a0 tx=0x7efc1800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:20.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.835+0000 7efc1cff9640 1 -- 172.21.15.83:0/980899778 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18017070 con 0x7efc2010e8e0 2024-04-25T11:54:20.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.835+0000 7efc25d19640 1 -- 172.21.15.83:0/980899778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2019b2b0 con 0x7efc2010e8e0 2024-04-25T11:54:20.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.835+0000 7efc1cff9640 1 -- 172.21.15.83:0/980899778 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc18009820 con 0x7efc2010e8e0 2024-04-25T11:54:20.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.835+0000 7efc1cff9640 1 -- 172.21.15.83:0/980899778 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18004c40 con 0x7efc2010e8e0 2024-04-25T11:54:20.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.835+0000 7efc25d19640 1 -- 172.21.15.83:0/980899778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc201a75c0 con 0x7efc2010e8e0 2024-04-25T11:54:20.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.836+0000 7efc25d19640 1 -- 172.21.15.83:0/980899778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc2019b5f0 con 0x7efc2010e8e0 2024-04-25T11:54:20.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.837+0000 7efc1cff9640 1 -- 172.21.15.83:0/980899778 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efc180099d0 con 0x7efc2010e8e0 2024-04-25T11:54:20.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.838+0000 7efc1cff9640 1 --2- 172.21.15.83:0/980899778 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7efc080a3d20 0x7efc080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:20.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.838+0000 7efc1f7fe640 1 --2- 172.21.15.83:0/980899778 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7efc080a3d20 0x7efc080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:20.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.838+0000 7efc1cff9640 1 -- 172.21.15.83:0/980899778 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7efc180d04b0 con 0x7efc2010e8e0 2024-04-25T11:54:20.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.838+0000 7efc1f7fe640 1 --2- 172.21.15.83:0/980899778 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7efc080a3d20 0x7efc080a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efc10007ec0 tx=0x7efc100075b0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:20.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:20.842+0000 7efc1cff9640 1 -- 172.21.15.83:0/980899778 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc1809cb50 con 0x7efc2010e8e0 2024-04-25T11:54:21.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.019+0000 7efc25d19640 1 -- 172.21.15.83:0/980899778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc2019bc30 con 0x7efc2010e8e0 2024-04-25T11:54:21.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.021+0000 7efc1cff9640 1 -- 172.21.15.83:0/980899778 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23048 (secure 0 0 0) 0x7efc1809c450 con 0x7efc2010e8e0 2024-04-25T11:54:21.010 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:54:21.010 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":99,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:54:16.743447+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:41:42.546991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:41:43.764226+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/1157946519":"2024-04-26T11:43:40.200248+0000","172.21.15.83:0/3934887058":"2024-04-26T11:43:40.200248+0000","172.21.15.83:0/457699907":"2024-04-26T11:43:40.200248+0000","172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1837678414":"2024-04-26T11:43:40.200248+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6800/1157946519":"2024-04-26T11:43:40.200248+0000","172.21.15.83:0/3642996010":"2024-04-26T11:43:40.200248+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:54:21.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.023+0000 7efc25d19640 1 -- 172.21.15.83:0/980899778 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7efc080a3d20 msgr2=0x7efc080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.023+0000 7efc25d19640 1 --2- 172.21.15.83:0/980899778 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7efc080a3d20 0x7efc080a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efc10007ec0 tx=0x7efc100075b0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.023+0000 7efc25d19640 1 -- 172.21.15.83:0/980899778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2010e8e0 msgr2=0x7efc2019a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.023+0000 7efc25d19640 1 --2- 172.21.15.83:0/980899778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2010e8e0 0x7efc2019a700 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efc1800d6a0 tx=0x7efc1800db60 comp rx=0 tx=0).stop 2024-04-25T11:54:21.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.023+0000 7efc25d19640 1 -- 172.21.15.83:0/980899778 shutdown_connections 2024-04-25T11:54:21.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.023+0000 7efc25d19640 1 --2- 172.21.15.83:0/980899778 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7efc080a3d20 0x7efc080a61e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.023+0000 7efc25d19640 1 --2- 172.21.15.83:0/980899778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2010e8e0 0x7efc2019a700 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.023+0000 7efc25d19640 1 --2- 172.21.15.83:0/980899778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efc20101650 0x7efc201a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.023+0000 7efc25d19640 1 --2- 172.21.15.83:0/980899778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc20100d30 0x7efc201a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.023+0000 7efc25d19640 1 -- 172.21.15.83:0/980899778 >> 172.21.15.83:0/980899778 conn(0x7efc200771d0 msgr2=0x7efc2010fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:21.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.023+0000 7efc25d19640 1 -- 172.21.15.83:0/980899778 shutdown_connections 2024-04-25T11:54:21.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.023+0000 7efc25d19640 1 -- 172.21.15.83:0/980899778 wait complete. 2024-04-25T11:54:21.023 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:37:15.914779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:41:42.546991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:41:43.764226+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:54:21.023 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:54:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.136+0000 7f5480872640 1 -- 172.21.15.83:0/3064992522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f547c10e830 msgr2=0x7f547c110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.136+0000 7f5480872640 1 --2- 172.21.15.83:0/3064992522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f547c10e830 0x7f547c110c20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5474009ca0 tx=0x7f547402a950 comp rx=0 tx=0).stop 2024-04-25T11:54:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.136+0000 7f5480872640 1 -- 172.21.15.83:0/3064992522 shutdown_connections 2024-04-25T11:54:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.136+0000 7f5480872640 1 --2- 172.21.15.83:0/3064992522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f547c10e830 0x7f547c110c20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.136+0000 7f5480872640 1 --2- 172.21.15.83:0/3064992522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f547c074b40 0x7f547c10e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.136+0000 7f5480872640 1 --2- 172.21.15.83:0/3064992522 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f547c074220 0x7f547c074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.136+0000 7f5480872640 1 -- 172.21.15.83:0/3064992522 >> 172.21.15.83:0/3064992522 conn(0x7f547c0793a0 msgr2=0x7f547c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.136+0000 7f5480872640 1 -- 172.21.15.83:0/3064992522 shutdown_connections 2024-04-25T11:54:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.136+0000 7f5480872640 1 -- 172.21.15.83:0/3064992522 wait complete. 2024-04-25T11:54:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f5480872640 1 Processor -- start 2024-04-25T11:54:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f5480872640 1 -- start start 2024-04-25T11:54:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f5480872640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f547c074220 0x7f547c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f5480872640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f547c074b40 0x7f547c1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:21.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f5480872640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f547c10e830 0x7f547c19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:21.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f5480872640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f547c1143c0 con 0x7f547c10e830 2024-04-25T11:54:21.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f5480872640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f547c114240 con 0x7f547c074b40 2024-04-25T11:54:21.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f5480872640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f547c114540 con 0x7f547c074220 2024-04-25T11:54:21.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f547ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f547c10e830 0x7f547c19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:21.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f547ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f547c10e830 0x7f547c19a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51268/0 (socket says 172.21.15.83:51268) 2024-04-25T11:54:21.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f547ad9d640 1 -- 172.21.15.83:0/1240269591 learned_addr learned my addr 172.21.15.83:0/1240269591 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:54:21.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f5479d9b640 1 --2- 172.21.15.83:0/1240269591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f547c074b40 0x7f547c1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:21.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f547a59c640 1 --2- 172.21.15.83:0/1240269591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f547c074220 0x7f547c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:21.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f547ad9d640 1 -- 172.21.15.83:0/1240269591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f547c074220 msgr2=0x7f547c1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f547ad9d640 1 --2- 172.21.15.83:0/1240269591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f547c074220 0x7f547c1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f547ad9d640 1 -- 172.21.15.83:0/1240269591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f547c074b40 msgr2=0x7f547c1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f547ad9d640 1 --2- 172.21.15.83:0/1240269591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f547c074b40 0x7f547c1a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f547ad9d640 1 -- 172.21.15.83:0/1240269591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c19afd0 con 0x7f547c10e830 2024-04-25T11:54:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.137+0000 7f547ad9d640 1 --2- 172.21.15.83:0/1240269591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f547c10e830 0x7f547c19a800 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f547402ae70 tx=0x7f547402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.138+0000 7f546b7fe640 1 -- 172.21.15.83:0/1240269591 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474034040 con 0x7f547c10e830 2024-04-25T11:54:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.138+0000 7f5480872640 1 -- 172.21.15.83:0/1240269591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547c19b260 con 0x7f547c10e830 2024-04-25T11:54:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.138+0000 7f546b7fe640 1 -- 172.21.15.83:0/1240269591 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54740308f0 con 0x7f547c10e830 2024-04-25T11:54:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.138+0000 7f546b7fe640 1 -- 172.21.15.83:0/1240269591 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54740047c0 con 0x7f547c10e830 2024-04-25T11:54:21.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.138+0000 7f5480872640 1 -- 172.21.15.83:0/1240269591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f547c1a74e0 con 0x7f547c10e830 2024-04-25T11:54:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.139+0000 7f5480872640 1 -- 172.21.15.83:0/1240269591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f547c075960 con 0x7f547c10e830 2024-04-25T11:54:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.140+0000 7f546b7fe640 1 -- 172.21.15.83:0/1240269591 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5474037020 con 0x7f547c10e830 2024-04-25T11:54:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.140+0000 7f546b7fe640 1 --2- 172.21.15.83:0/1240269591 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f54640a3d20 0x7f54640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.140+0000 7f547a59c640 1 --2- 172.21.15.83:0/1240269591 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f54640a3d20 0x7f54640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.140+0000 7f546b7fe640 1 -- 172.21.15.83:0/1240269591 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f54740f16c0 con 0x7f547c10e830 2024-04-25T11:54:21.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.141+0000 7f547a59c640 1 --2- 172.21.15.83:0/1240269591 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f54640a3d20 0x7f54640a61e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5470005fd0 tx=0x7f547000b590 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:21.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.145+0000 7f546b7fe640 1 -- 172.21.15.83:0/1240269591 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54740bde70 con 0x7f547c10e830 2024-04-25T11:54:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.324+0000 7f5480872640 1 -- 172.21.15.83:0/1240269591 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f547c1099b0 con 0x7f547c10e830 2024-04-25T11:54:21.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.325+0000 7f546b7fe640 1 -- 172.21.15.83:0/1240269591 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f54740bd770 con 0x7f547c10e830 2024-04-25T11:54:21.314 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-25T11:54:21.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.327+0000 7f5480872640 1 -- 172.21.15.83:0/1240269591 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f54640a3d20 msgr2=0x7f54640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.327+0000 7f5480872640 1 --2- 172.21.15.83:0/1240269591 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f54640a3d20 0x7f54640a61e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5470005fd0 tx=0x7f547000b590 comp rx=0 tx=0).stop 2024-04-25T11:54:21.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.327+0000 7f5480872640 1 -- 172.21.15.83:0/1240269591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f547c10e830 msgr2=0x7f547c19a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.327+0000 7f5480872640 1 --2- 172.21.15.83:0/1240269591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f547c10e830 0x7f547c19a800 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f547402ae70 tx=0x7f547402f5d0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.327+0000 7f5480872640 1 -- 172.21.15.83:0/1240269591 shutdown_connections 2024-04-25T11:54:21.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.327+0000 7f5480872640 1 --2- 172.21.15.83:0/1240269591 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f54640a3d20 0x7f54640a61e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.327+0000 7f5480872640 1 --2- 172.21.15.83:0/1240269591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f547c10e830 0x7f547c19a800 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.327+0000 7f5480872640 1 --2- 172.21.15.83:0/1240269591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f547c074b40 0x7f547c1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.327+0000 7f5480872640 1 --2- 172.21.15.83:0/1240269591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f547c074220 0x7f547c1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.327+0000 7f5480872640 1 -- 172.21.15.83:0/1240269591 >> 172.21.15.83:0/1240269591 conn(0x7f547c0793a0 msgr2=0x7f547c1002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:21.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.327+0000 7f5480872640 1 -- 172.21.15.83:0/1240269591 shutdown_connections 2024-04-25T11:54:21.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.327+0000 7f5480872640 1 -- 172.21.15.83:0/1240269591 wait complete. 2024-04-25T11:54:21.327 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:54:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.439+0000 7fe683f07640 1 -- 172.21.15.83:0/305757514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c1017d0 msgr2=0x7fe67c101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.439+0000 7fe683f07640 1 --2- 172.21.15.83:0/305757514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c1017d0 0x7fe67c101bb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009900 tx=0x7fe66c02a410 comp rx=0 tx=0).stop 2024-04-25T11:54:21.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.439+0000 7fe683f07640 1 -- 172.21.15.83:0/305757514 shutdown_connections 2024-04-25T11:54:21.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.439+0000 7fe683f07640 1 --2- 172.21.15.83:0/305757514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe67c106380 0x7fe67c10df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.439+0000 7fe683f07640 1 --2- 172.21.15.83:0/305757514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe67c102180 0x7fe67c105ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.439+0000 7fe683f07640 1 --2- 172.21.15.83:0/305757514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c1017d0 0x7fe67c101bb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.439+0000 7fe683f07640 1 -- 172.21.15.83:0/305757514 >> 172.21.15.83:0/305757514 conn(0x7fe67c077410 msgr2=0x7fe67c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:21.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.440+0000 7fe683f07640 1 -- 172.21.15.83:0/305757514 shutdown_connections 2024-04-25T11:54:21.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.440+0000 7fe683f07640 1 -- 172.21.15.83:0/305757514 wait complete. 2024-04-25T11:54:21.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.440+0000 7fe683f07640 1 Processor -- start 2024-04-25T11:54:21.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.440+0000 7fe683f07640 1 -- start start 2024-04-25T11:54:21.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.440+0000 7fe683f07640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c1017d0 0x7fe67c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:21.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.440+0000 7fe683f07640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe67c106380 0x7fe67c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:21.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.440+0000 7fe683f07640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe67c198780 0x7fe67c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:21.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.440+0000 7fe683f07640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c112440 con 0x7fe67c1017d0 2024-04-25T11:54:21.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.440+0000 7fe683f07640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c1122c0 con 0x7fe67c106380 2024-04-25T11:54:21.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.440+0000 7fe683f07640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c1125c0 con 0x7fe67c198780 2024-04-25T11:54:21.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.440+0000 7fe681ca3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c1017d0 0x7fe67c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:21.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.440+0000 7fe681ca3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c1017d0 0x7fe67c19e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51280/0 (socket says 172.21.15.83:51280) 2024-04-25T11:54:21.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.440+0000 7fe681ca3640 1 -- 172.21.15.83:0/2938427096 learned_addr learned my addr 172.21.15.83:0/2938427096 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:54:21.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.441+0000 7fe6824a4640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe67c198780 0x7fe67c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:21.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.441+0000 7fe681ca3640 1 -- 172.21.15.83:0/2938427096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe67c198780 msgr2=0x7fe67c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.441+0000 7fe681ca3640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe67c198780 0x7fe67c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.441+0000 7fe681ca3640 1 -- 172.21.15.83:0/2938427096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe67c106380 msgr2=0x7fe67c19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.441+0000 7fe6814a2640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe67c106380 0x7fe67c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.441+0000 7fe681ca3640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe67c106380 0x7fe67c19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.441+0000 7fe681ca3640 1 -- 172.21.15.83:0/2938427096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67c199310 con 0x7fe67c1017d0 2024-04-25T11:54:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.441+0000 7fe6824a4640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe67c198780 0x7fe67c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:54:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.441+0000 7fe6814a2640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe67c106380 0x7fe67c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:54:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.441+0000 7fe681ca3640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c1017d0 0x7fe67c19e610 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe66c006010 tx=0x7fe66c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.441+0000 7fe672ffd640 1 -- 172.21.15.83:0/2938427096 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c03f030 con 0x7fe67c1017d0 2024-04-25T11:54:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.441+0000 7fe672ffd640 1 -- 172.21.15.83:0/2938427096 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe66c003800 con 0x7fe67c1017d0 2024-04-25T11:54:21.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.441+0000 7fe683f07640 1 -- 172.21.15.83:0/2938427096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe67c1995a0 con 0x7fe67c1017d0 2024-04-25T11:54:21.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.441+0000 7fe672ffd640 1 -- 172.21.15.83:0/2938427096 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c035070 con 0x7fe67c1017d0 2024-04-25T11:54:21.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.441+0000 7fe683f07640 1 -- 172.21.15.83:0/2938427096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe67c1a5750 con 0x7fe67c1017d0 2024-04-25T11:54:21.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.442+0000 7fe683f07640 1 -- 172.21.15.83:0/2938427096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe67c102810 con 0x7fe67c1017d0 2024-04-25T11:54:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.443+0000 7fe672ffd640 1 -- 172.21.15.83:0/2938427096 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe66c003cb0 con 0x7fe67c1017d0 2024-04-25T11:54:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.444+0000 7fe672ffd640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fe6680a3d20 0x7fe6680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.444+0000 7fe6814a2640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fe6680a3d20 0x7fe6680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.444+0000 7fe672ffd640 1 -- 172.21.15.83:0/2938427096 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fe66c0f14b0 con 0x7fe67c1017d0 2024-04-25T11:54:21.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.444+0000 7fe6814a2640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fe6680a3d20 0x7fe6680a61e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0ffa80 tx=0x7fe678005d30 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:21.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.447+0000 7fe672ffd640 1 -- 172.21.15.83:0/2938427096 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe66c0bdb50 con 0x7fe67c1017d0 2024-04-25T11:54:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.626+0000 7fe683f07640 1 -- 172.21.15.83:0/2938427096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe67c0691c0 con 0x7fe67c1017d0 2024-04-25T11:54:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.626+0000 7fe672ffd640 1 -- 172.21.15.83:0/2938427096 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7fe67c0691c0 con 0x7fe67c1017d0 2024-04-25T11:54:21.615 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-25T11:54:21.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.628+0000 7fe683f07640 1 -- 172.21.15.83:0/2938427096 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fe6680a3d20 msgr2=0x7fe6680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.628+0000 7fe683f07640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fe6680a3d20 0x7fe6680a61e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0ffa80 tx=0x7fe678005d30 comp rx=0 tx=0).stop 2024-04-25T11:54:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.628+0000 7fe683f07640 1 -- 172.21.15.83:0/2938427096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c1017d0 msgr2=0x7fe67c19e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.628+0000 7fe683f07640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c1017d0 0x7fe67c19e610 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe66c006010 tx=0x7fe66c0042f0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.628+0000 7fe683f07640 1 -- 172.21.15.83:0/2938427096 shutdown_connections 2024-04-25T11:54:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.628+0000 7fe683f07640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fe6680a3d20 0x7fe6680a61e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.628+0000 7fe683f07640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe67c198780 0x7fe67c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.628+0000 7fe683f07640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe67c106380 0x7fe67c19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.628+0000 7fe683f07640 1 --2- 172.21.15.83:0/2938427096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe67c1017d0 0x7fe67c19e610 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.628+0000 7fe683f07640 1 -- 172.21.15.83:0/2938427096 >> 172.21.15.83:0/2938427096 conn(0x7fe67c077410 msgr2=0x7fe67c104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.628+0000 7fe683f07640 1 -- 172.21.15.83:0/2938427096 shutdown_connections 2024-04-25T11:54:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.628+0000 7fe683f07640 1 -- 172.21.15.83:0/2938427096 wait complete. 2024-04-25T11:54:21.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:54:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.739+0000 7f4d0855d640 1 -- 172.21.15.83:0/514860381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d0006a7c0 msgr2=0x7f4d0006aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.739+0000 7f4d0855d640 1 --2- 172.21.15.83:0/514860381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d0006a7c0 0x7f4d0006aba0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009900 tx=0x7f4cf002a410 comp rx=0 tx=0).stop 2024-04-25T11:54:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.739+0000 7f4d0855d640 1 -- 172.21.15.83:0/514860381 shutdown_connections 2024-04-25T11:54:21.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.739+0000 7f4d0855d640 1 --2- 172.21.15.83:0/514860381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d00107280 0x7f4d0010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.739+0000 7f4d0855d640 1 --2- 172.21.15.83:0/514860381 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d0006b170 0x7f4d00106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.739+0000 7f4d0855d640 1 --2- 172.21.15.83:0/514860381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d0006a7c0 0x7f4d0006aba0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.739+0000 7f4d0855d640 1 -- 172.21.15.83:0/514860381 >> 172.21.15.83:0/514860381 conn(0x7f4d00076c50 msgr2=0x7f4d00077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.739+0000 7f4d0855d640 1 -- 172.21.15.83:0/514860381 shutdown_connections 2024-04-25T11:54:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.739+0000 7f4d0855d640 1 -- 172.21.15.83:0/514860381 wait complete. 2024-04-25T11:54:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d0855d640 1 Processor -- start 2024-04-25T11:54:21.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d0855d640 1 -- start start 2024-04-25T11:54:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d0855d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d0006a7c0 0x7f4d00198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d0855d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d00107280 0x7f4d00198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d0855d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d0019d000 0x7f4d0019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d0855d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d00112220 con 0x7f4d00107280 2024-04-25T11:54:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d0855d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001120a0 con 0x7f4d0019d000 2024-04-25T11:54:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d0855d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001123a0 con 0x7f4d0006a7c0 2024-04-25T11:54:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d05af8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d00107280 0x7f4d00198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:21.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d05af8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d00107280 0x7f4d00198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51296/0 (socket says 172.21.15.83:51296) 2024-04-25T11:54:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d05af8640 1 -- 172.21.15.83:0/3253243041 learned_addr learned my addr 172.21.15.83:0/3253243041 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:54:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d062f9640 1 --2- 172.21.15.83:0/3253243041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d0006a7c0 0x7f4d00198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d06afa640 1 --2- 172.21.15.83:0/3253243041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d0019d000 0x7f4d0019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d05af8640 1 -- 172.21.15.83:0/3253243041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d0006a7c0 msgr2=0x7f4d00198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d05af8640 1 --2- 172.21.15.83:0/3253243041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d0006a7c0 0x7f4d00198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d05af8640 1 -- 172.21.15.83:0/3253243041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d0019d000 msgr2=0x7f4d0019d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d05af8640 1 --2- 172.21.15.83:0/3253243041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d0019d000 0x7f4d0019d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.740+0000 7f4d05af8640 1 -- 172.21.15.83:0/3253243041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0019daf0 con 0x7f4d00107280 2024-04-25T11:54:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.741+0000 7f4d05af8640 1 --2- 172.21.15.83:0/3253243041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d00107280 0x7f4d00198c80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc006fd0 tx=0x7f4cfc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.741+0000 7f4cf77fe640 1 -- 172.21.15.83:0/3253243041 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc011830 con 0x7f4d00107280 2024-04-25T11:54:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.741+0000 7f4d0855d640 1 -- 172.21.15.83:0/3253243041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0019dde0 con 0x7f4d00107280 2024-04-25T11:54:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.741+0000 7f4cf77fe640 1 -- 172.21.15.83:0/3253243041 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cfc016070 con 0x7f4d00107280 2024-04-25T11:54:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.741+0000 7f4cf77fe640 1 -- 172.21.15.83:0/3253243041 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc010e00 con 0x7f4d00107280 2024-04-25T11:54:21.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.741+0000 7f4d0855d640 1 -- 172.21.15.83:0/3253243041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d001a5650 con 0x7f4d00107280 2024-04-25T11:54:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.742+0000 7f4d0855d640 1 -- 172.21.15.83:0/3253243041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d00103680 con 0x7f4d00107280 2024-04-25T11:54:21.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.743+0000 7f4cf77fe640 1 -- 172.21.15.83:0/3253243041 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4cfc01b460 con 0x7f4d00107280 2024-04-25T11:54:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.743+0000 7f4cf77fe640 1 --2- 172.21.15.83:0/3253243041 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f4cec0a3d20 0x7f4cec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.743+0000 7f4d062f9640 1 --2- 172.21.15.83:0/3253243041 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f4cec0a3d20 0x7f4cec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.744+0000 7f4cf77fe640 1 -- 172.21.15.83:0/3253243041 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f4cfc0d0360 con 0x7f4d00107280 2024-04-25T11:54:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.744+0000 7f4d062f9640 1 --2- 172.21.15.83:0/3253243041 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f4cec0a3d20 0x7f4cec0a61e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0031440 tx=0x7f4cf0005b50 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:21.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.747+0000 7f4cf77fe640 1 -- 172.21.15.83:0/3253243041 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4cfc09cb10 con 0x7f4d00107280 2024-04-25T11:54:21.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:21 smithi083 ceph-mon[29330]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 210 B/s wr, 10 op/s 2024-04-25T11:54:21.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:21 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/980899778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:54:21.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:21 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1240269591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:54:21.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:21 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2938427096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:54:21.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.928+0000 7f4d0855d640 1 -- 172.21.15.83:0/3253243041 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4d00199a90 con 0x7f4d00107280 2024-04-25T11:54:21.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.929+0000 7f4cf77fe640 1 -- 172.21.15.83:0/3253243041 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f4cfc09c410 con 0x7f4d00107280 2024-04-25T11:54:21.917 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-25T11:54:21.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.930+0000 7f4d0855d640 1 -- 172.21.15.83:0/3253243041 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f4cec0a3d20 msgr2=0x7f4cec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.930+0000 7f4d0855d640 1 --2- 172.21.15.83:0/3253243041 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f4cec0a3d20 0x7f4cec0a61e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0031440 tx=0x7f4cf0005b50 comp rx=0 tx=0).stop 2024-04-25T11:54:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.930+0000 7f4d0855d640 1 -- 172.21.15.83:0/3253243041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d00107280 msgr2=0x7f4d00198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.930+0000 7f4d0855d640 1 --2- 172.21.15.83:0/3253243041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d00107280 0x7f4d00198c80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc006fd0 tx=0x7f4cfc00b8e0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.931+0000 7f4d0855d640 1 -- 172.21.15.83:0/3253243041 shutdown_connections 2024-04-25T11:54:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.931+0000 7f4d0855d640 1 --2- 172.21.15.83:0/3253243041 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f4cec0a3d20 0x7f4cec0a61e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.931+0000 7f4d0855d640 1 --2- 172.21.15.83:0/3253243041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d00107280 0x7f4d00198c80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.931+0000 7f4d0855d640 1 --2- 172.21.15.83:0/3253243041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d0019d000 0x7f4d0019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.931+0000 7f4d0855d640 1 --2- 172.21.15.83:0/3253243041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d0006a7c0 0x7f4d00198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.931+0000 7f4d0855d640 1 -- 172.21.15.83:0/3253243041 >> 172.21.15.83:0/3253243041 conn(0x7f4d00076c50 msgr2=0x7f4d000ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.931+0000 7f4d0855d640 1 -- 172.21.15.83:0/3253243041 shutdown_connections 2024-04-25T11:54:21.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:21.931+0000 7f4d0855d640 1 -- 172.21.15.83:0/3253243041 wait complete. 2024-04-25T11:54:21.930 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:54:22.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:21 smithi094 ceph-mon[33032]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 210 B/s wr, 10 op/s 2024-04-25T11:54:22.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:21 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/980899778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:54:22.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:21 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1240269591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:54:22.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:21 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2938427096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:54:22.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.040+0000 7fe8c58ae640 1 -- 172.21.15.83:0/2045202780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0101aa0 msgr2=0x7fe8c010dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:22.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.040+0000 7fe8c58ae640 1 --2- 172.21.15.83:0/2045202780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0101aa0 0x7fe8c010dcb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009900 tx=0x7fe8b802a470 comp rx=0 tx=0).stop 2024-04-25T11:54:22.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.040+0000 7fe8c58ae640 1 -- 172.21.15.83:0/2045202780 shutdown_connections 2024-04-25T11:54:22.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.040+0000 7fe8c58ae640 1 --2- 172.21.15.83:0/2045202780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0101aa0 0x7fe8c010dcb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.040+0000 7fe8c58ae640 1 --2- 172.21.15.83:0/2045202780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8c01010e0 0x7fe8c0101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.040+0000 7fe8c58ae640 1 --2- 172.21.15.83:0/2045202780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c0105e70 0x7fe8c0106250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.040+0000 7fe8c58ae640 1 -- 172.21.15.83:0/2045202780 >> 172.21.15.83:0/2045202780 conn(0x7fe8c0077cb0 msgr2=0x7fe8c00780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:22.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.040+0000 7fe8c58ae640 1 -- 172.21.15.83:0/2045202780 shutdown_connections 2024-04-25T11:54:22.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.041+0000 7fe8c58ae640 1 -- 172.21.15.83:0/2045202780 wait complete. 2024-04-25T11:54:22.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.041+0000 7fe8c58ae640 1 Processor -- start 2024-04-25T11:54:22.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.041+0000 7fe8c58ae640 1 -- start start 2024-04-25T11:54:22.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.041+0000 7fe8c58ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c01010e0 0x7fe8c01994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:22.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.041+0000 7fe8c58ae640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8c0101aa0 0x7fe8c0199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:22.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.041+0000 7fe8c58ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0105e70 0x7fe8c01a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:22.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.041+0000 7fe8c58ae640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c0111110 con 0x7fe8c0105e70 2024-04-25T11:54:22.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.041+0000 7fe8c58ae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c0110f90 con 0x7fe8c01010e0 2024-04-25T11:54:22.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.041+0000 7fe8c58ae640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c0111290 con 0x7fe8c0101aa0 2024-04-25T11:54:22.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.041+0000 7fe8bf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0105e70 0x7fe8c01a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:22.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.041+0000 7fe8bf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0105e70 0x7fe8c01a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51316/0 (socket says 172.21.15.83:51316) 2024-04-25T11:54:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8bf7fe640 1 -- 172.21.15.83:0/2152735809 learned_addr learned my addr 172.21.15.83:0/2152735809 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:54:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8be7fc640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8c0101aa0 0x7fe8c0199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8beffd640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c01010e0 0x7fe8c01994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8bf7fe640 1 -- 172.21.15.83:0/2152735809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8c0101aa0 msgr2=0x7fe8c0199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8bf7fe640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8c0101aa0 0x7fe8c0199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8bf7fe640 1 -- 172.21.15.83:0/2152735809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c01010e0 msgr2=0x7fe8c01994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8bf7fe640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c01010e0 0x7fe8c01994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8bf7fe640 1 -- 172.21.15.83:0/2152735809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c01a4ab0 con 0x7fe8c0105e70 2024-04-25T11:54:22.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8be7fc640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8c0101aa0 0x7fe8c0199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:54:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8beffd640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c01010e0 0x7fe8c01994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:54:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8bf7fe640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0105e70 0x7fe8c01a4570 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8005ab0 tx=0x7fe8b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8c48ac640 1 -- 172.21.15.83:0/2152735809 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b803f030 con 0x7fe8c0105e70 2024-04-25T11:54:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8c58ae640 1 -- 172.21.15.83:0/2152735809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c01a4c80 con 0x7fe8c0105e70 2024-04-25T11:54:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8c48ac640 1 -- 172.21.15.83:0/2152735809 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b8003cb0 con 0x7fe8c0105e70 2024-04-25T11:54:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8c48ac640 1 -- 172.21.15.83:0/2152735809 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8035070 con 0x7fe8c0105e70 2024-04-25T11:54:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.042+0000 7fe8c58ae640 1 -- 172.21.15.83:0/2152735809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c01a50e0 con 0x7fe8c0105e70 2024-04-25T11:54:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.043+0000 7fe8c58ae640 1 -- 172.21.15.83:0/2152735809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c0068d70 con 0x7fe8c0105e70 2024-04-25T11:54:22.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.044+0000 7fe8c48ac640 1 -- 172.21.15.83:0/2152735809 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe8b8042030 con 0x7fe8c0105e70 2024-04-25T11:54:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.045+0000 7fe8c48ac640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fe8a80a3d20 0x7fe8a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.045+0000 7fe8beffd640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fe8a80a3d20 0x7fe8a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:22.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.045+0000 7fe8c48ac640 1 -- 172.21.15.83:0/2152735809 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fe8b80f12e0 con 0x7fe8c0105e70 2024-04-25T11:54:22.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.045+0000 7fe8beffd640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fe8a80a3d20 0x7fe8a80a61e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0005fd0 tx=0x7fe8b00082b0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:22.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.049+0000 7fe8c48ac640 1 -- 172.21.15.83:0/2152735809 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8b80bd980 con 0x7fe8c0105e70 2024-04-25T11:54:22.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:21 smithi111 ceph-mon[32784]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 210 B/s wr, 10 op/s 2024-04-25T11:54:22.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:21 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/980899778' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:54:22.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:21 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1240269591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T11:54:22.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:21 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2938427096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T11:54:22.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.240+0000 7fe8c58ae640 1 -- 172.21.15.83:0/2152735809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe8c002d5c0 con 0x7fe8c0105e70 2024-04-25T11:54:22.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.240+0000 7fe8c48ac640 1 -- 172.21.15.83:0/2152735809 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v99) ==== 81+0+943 (secure 0 0 0) 0x7fe8b80bd280 con 0x7fe8c0105e70 2024-04-25T11:54:22.229 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/1157946519 2024-04-26T11:43:40.200248+0000 2024-04-25T11:54:22.229 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3934887058 2024-04-26T11:43:40.200248+0000 2024-04-25T11:54:22.229 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/457699907 2024-04-26T11:43:40.200248+0000 2024-04-25T11:54:22.229 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/486441637 2024-04-26T11:34:49.826109+0000 2024-04-25T11:54:22.230 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1837678414 2024-04-26T11:43:40.200248+0000 2024-04-25T11:54:22.230 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1048355479 2024-04-26T11:34:49.826109+0000 2024-04-25T11:54:22.230 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/1639787889 2024-04-26T11:34:49.826109+0000 2024-04-25T11:54:22.230 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/1639787889 2024-04-26T11:34:49.826109+0000 2024-04-25T11:54:22.230 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3257452011 2024-04-26T11:34:23.164299+0000 2024-04-25T11:54:22.230 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/908199980 2024-04-26T11:34:49.826109+0000 2024-04-25T11:54:22.230 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/3997740373 2024-04-26T11:34:23.164299+0000 2024-04-25T11:54:22.230 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/2677315213 2024-04-26T11:34:23.164299+0000 2024-04-25T11:54:22.230 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/1157946519 2024-04-26T11:43:40.200248+0000 2024-04-25T11:54:22.231 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3642996010 2024-04-26T11:43:40.200248+0000 2024-04-25T11:54:22.231 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/3997740373 2024-04-26T11:34:23.164299+0000 2024-04-25T11:54:22.231 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1174617340 2024-04-26T11:34:23.164299+0000 2024-04-25T11:54:22.232 INFO:teuthology.orchestra.run.smithi083.stderr:listed 16 entries 2024-04-25T11:54:22.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.242+0000 7fe8c58ae640 1 -- 172.21.15.83:0/2152735809 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fe8a80a3d20 msgr2=0x7fe8a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:22.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.242+0000 7fe8c58ae640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fe8a80a3d20 0x7fe8a80a61e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0005fd0 tx=0x7fe8b00082b0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.242+0000 7fe8c58ae640 1 -- 172.21.15.83:0/2152735809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0105e70 msgr2=0x7fe8c01a4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:22.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.242+0000 7fe8c58ae640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0105e70 0x7fe8c01a4570 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8005ab0 tx=0x7fe8b80042f0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.242+0000 7fe8c58ae640 1 -- 172.21.15.83:0/2152735809 shutdown_connections 2024-04-25T11:54:22.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.242+0000 7fe8c58ae640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fe8a80a3d20 0x7fe8a80a61e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.242+0000 7fe8c58ae640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0105e70 0x7fe8c01a4570 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.242+0000 7fe8c58ae640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8c0101aa0 0x7fe8c0199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.242+0000 7fe8c58ae640 1 --2- 172.21.15.83:0/2152735809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c01010e0 0x7fe8c01994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.242+0000 7fe8c58ae640 1 -- 172.21.15.83:0/2152735809 >> 172.21.15.83:0/2152735809 conn(0x7fe8c0077cb0 msgr2=0x7fe8c00ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:22.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.242+0000 7fe8c58ae640 1 -- 172.21.15.83:0/2152735809 shutdown_connections 2024-04-25T11:54:22.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.242+0000 7fe8c58ae640 1 -- 172.21.15.83:0/2152735809 wait complete. 2024-04-25T11:54:22.241 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:54:22.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:54:22.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:54:22.270 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T11:54:22.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:54:22.270 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:54:22.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:54:22.337 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T11:54:22.392 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:54:22.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:54:22.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:54:22.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:54:22.458 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-25T11:54:22.560 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T11:54:22.560 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-25T11:54:22.560 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:54:22.576 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056106,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20524 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/b853a8aa880317e2bf315f681f79b5247d093c179f4c8f782d5ec0cf970733ed/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/RGOC6RUXH25VENNLYSNU5TEYH3:/var/lib/containers/storage/overlay/l/CLCZZURKWD4RNIZKDPSL6KP6AO,upperdir=/var/lib/containers/storage/overlay/b853a8aa880317e2bf315f681f79b5247d093c179f4c8f782d5ec0cf970733ed/diff,workdir=/var/lib/containers/storage/overlay/b853a8aa880317e2bf315f681f79b5247d093c179f4c8f782d5ec0cf970733ed/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e2548cfbe546b8edb57adb9038670ac12d2027b32f480d8fc59985a3edfef24c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/RGOC6RUXH25VENNLYSNU5TEYH3:/var/lib/containers/storage/overlay/l/CLCZZURKWD4RNIZKDPSL6KP6AO,upperdir=/var/lib/containers/storage/overlay/e2548cfbe546b8edb57adb9038670ac12d2027b32f480d8fc59985a3edfef24c/diff,workdir=/var/lib/containers/storage/overlay/e2548cfbe546b8edb57adb9038670ac12d2027b32f480d8fc59985a3edfef24c/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/a11c6e24b65c0822a56003c80158d8939cb774e23832e28286117561ec58aa4d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/RGOC6RUXH25VENNLYSNU5TEYH3:/var/lib/containers/storage/overlay/l/CLCZZURKWD4RNIZKDPSL6KP6AO,upperdir=/var/lib/containers/storage/overlay/a11c6e24b65c0822a56003c80158d8939cb774e23832e28286117561ec58aa4d/diff,workdir=/var/lib/containers/storage/overlay/a11c6e24b65c0822a56003c80158d8939cb774e23832e28286117561ec58aa4d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6f40c57f0829eec36f8e300dbb7e0efd72cb2eda56b3e5e95240eedc28295931/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/RGOC6RUXH25VENNLYSNU5TEYH3:/var/lib/containers/storage/overlay/l/CLCZZURKWD4RNIZKDPSL6KP6AO,upperdir=/var/lib/containers/storage/overlay/6f40c57f0829eec36f8e300dbb7e0efd72cb2eda56b3e5e95240eedc28295931/diff,workdir=/var/lib/containers/storage/overlay/6f40c57f0829eec36f8e300dbb7e0efd72cb2eda56b3e5e95240eedc28295931/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a61b0e37e98f8c388489bd665f1fed2c260238afecb1b518f9f860b16222f466/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/RGOC6RUXH25VENNLYSNU5TEYH3:/var/lib/containers/storage/overlay/l/CLCZZURKWD4RNIZKDPSL6KP6AO,upperdir=/var/lib/containers/storage/overlay/a61b0e37e98f8c388489bd665f1fed2c260238afecb1b518f9f860b16222f466/diff,workdir=/var/lib/containers/storage/overlay/a61b0e37e98f8c388489bd665f1fed2c260238afecb1b518f9f860b16222f466/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6eef7cf4ae8ac213029581e5bcdc464dfcf20ca18088596e071c0cfc2bcff549/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/RGOC6RUXH25VENNLYSNU5TEYH3:/var/lib/containers/storage/overlay/l/CLCZZURKWD4RNIZKDPSL6KP6AO,upperdir=/var/lib/containers/storage/overlay/6eef7cf4ae8ac213029581e5bcdc464dfcf20ca18088596e071c0cfc2bcff549/diff,workdir=/var/lib/containers/storage/overlay/6eef7cf4ae8ac213029581e5bcdc464dfcf20ca18088596e071c0cfc2bcff549/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/db6b0ea39636768c2a1f614295db915dd7cc416633ece0b6e32182c8805d11ae/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/RGOC6RUXH25VENNLYSNU5TEYH3:/var/lib/containers/storage/overlay/l/CLCZZURKWD4RNIZKDPSL6KP6AO,upperdir=/var/lib/containers/storage/overlay/db6b0ea39636768c2a1f614295db915dd7cc416633ece0b6e32182c8805d11ae/diff,workdir=/var/lib/containers/storage/overlay/db6b0ea39636768c2a1f614295db915dd7cc416633ece0b6e32182c8805d11ae/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5858be84fd6e75d8c6cf3e3be88f535929a133c800de2187d73a936f4432f02a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/RGOC6RUXH25VENNLYSNU5TEYH3:/var/lib/containers/storage/overlay/l/CLCZZURKWD4RNIZKDPSL6KP6AO,upperdir=/var/lib/containers/storage/overlay/5858be84fd6e75d8c6cf3e3be88f535929a133c800de2187d73a936f4432f02a/diff,workdir=/var/lib/containers/storage/overlay/5858be84fd6e75d8c6cf3e3be88f535929a133c800de2187d73a936f4432f02a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/62378789f3481133a61741e79f9e82df84acaeead4fd652a9b8908ad9011fe6f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/RGOC6RUXH25VENNLYSNU5TEYH3:/var/lib/containers/storage/overlay/l/CLCZZURKWD4RNIZKDPSL6KP6AO,upperdir=/var/lib/containers/storage/overlay/62378789f3481133a61741e79f9e82df84acaeead4fd652a9b8908ad9011fe6f/diff,workdir=/var/lib/containers/storage/overlay/62378789f3481133a61741e79f9e82df84acaeead4fd652a9b8908ad9011fe6f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0f68ac97f2279289a373514013c9139dd07d0597b74288aa937409df02a651e3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/RGOC6RUXH25VENNLYSNU5TEYH3:/var/lib/containers/storage/overlay/l/CLCZZURKWD4RNIZKDPSL6KP6AO,upperdir=/var/lib/containers/storage/overlay/0f68ac97f2279289a373514013c9139dd07d0597b74288aa937409df02a651e3/diff,workdir=/var/lib/containers/storage/overlay/0f68ac97f2279289a373514013c9139dd07d0597b74288aa937409df02a651e3/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-25T11:54:22.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:54:22.576 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-25T11:54:22.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:54:22.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-25T11:54:22.699 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2024-04-25T11:54:22.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:54:22.700 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T11:54:22.711 INFO:tasks.cephadm:Teardown begin 2024-04-25T11:54:22.712 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:54:22.712 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:54:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.850+0000 7f3fe2ec2640 1 -- 172.21.15.83:0/3115979969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fdc1010a0 msgr2=0x7f3fdc101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.850+0000 7f3fe2ec2640 1 --2- 172.21.15.83:0/3115979969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fdc1010a0 0x7f3fdc101480 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009900 tx=0x7f3fd002a410 comp rx=0 tx=0).stop 2024-04-25T11:54:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.850+0000 7f3fe2ec2640 1 -- 172.21.15.83:0/3115979969 shutdown_connections 2024-04-25T11:54:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.850+0000 7f3fe2ec2640 1 --2- 172.21.15.83:0/3115979969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fdc106170 0x7f3fdc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.850+0000 7f3fe2ec2640 1 --2- 172.21.15.83:0/3115979969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fdc101a50 0x7f3fdc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.850+0000 7f3fe2ec2640 1 --2- 172.21.15.83:0/3115979969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fdc1010a0 0x7f3fdc101480 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.850+0000 7f3fe2ec2640 1 -- 172.21.15.83:0/3115979969 >> 172.21.15.83:0/3115979969 conn(0x7f3fdc077f50 msgr2=0x7f3fdc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.850+0000 7f3fe2ec2640 1 -- 172.21.15.83:0/3115979969 shutdown_connections 2024-04-25T11:54:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.850+0000 7f3fe2ec2640 1 -- 172.21.15.83:0/3115979969 wait complete. 2024-04-25T11:54:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fe2ec2640 1 Processor -- start 2024-04-25T11:54:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fe2ec2640 1 -- start start 2024-04-25T11:54:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fe2ec2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fdc1010a0 0x7f3fdc197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fe2ec2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fdc106170 0x7f3fdc197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fe2ec2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fdc19bee0 0x7f3fdc19c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fe2ec2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc111130 con 0x7f3fdc106170 2024-04-25T11:54:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fe2ec2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc110fb0 con 0x7f3fdc1010a0 2024-04-25T11:54:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fe2ec2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc1112b0 con 0x7f3fdc19bee0 2024-04-25T11:54:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fdbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fdc106170 0x7f3fdc197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fdbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fdc106170 0x7f3fdc197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37940/0 (socket says 172.21.15.83:37940) 2024-04-25T11:54:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fdbfff640 1 -- 172.21.15.83:0/253094064 learned_addr learned my addr 172.21.15.83:0/253094064 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:54:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fe0c5e640 1 --2- 172.21.15.83:0/253094064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fdc1010a0 0x7f3fdc197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fe145f640 1 --2- 172.21.15.83:0/253094064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fdc19bee0 0x7f3fdc19c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fdbfff640 1 -- 172.21.15.83:0/253094064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fdc19bee0 msgr2=0x7f3fdc19c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fdbfff640 1 --2- 172.21.15.83:0/253094064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fdc19bee0 0x7f3fdc19c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fdbfff640 1 -- 172.21.15.83:0/253094064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fdc1010a0 msgr2=0x7f3fdc197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fdbfff640 1 --2- 172.21.15.83:0/253094064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fdc1010a0 0x7f3fdc197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.851+0000 7f3fdbfff640 1 -- 172.21.15.83:0/253094064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fdc19c9d0 con 0x7f3fdc106170 2024-04-25T11:54:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.852+0000 7f3fdbfff640 1 --2- 172.21.15.83:0/253094064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fdc106170 0x7f3fdc197bb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3fc800c970 tx=0x7f3fc800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.852+0000 7f3fd9ffb640 1 -- 172.21.15.83:0/253094064 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8014070 con 0x7f3fdc106170 2024-04-25T11:54:22.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.852+0000 7f3fd9ffb640 1 -- 172.21.15.83:0/253094064 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fc8004440 con 0x7f3fdc106170 2024-04-25T11:54:22.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.852+0000 7f3fe2ec2640 1 -- 172.21.15.83:0/253094064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fdc19ccc0 con 0x7f3fdc106170 2024-04-25T11:54:22.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.852+0000 7f3fd9ffb640 1 -- 172.21.15.83:0/253094064 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8003e50 con 0x7f3fdc106170 2024-04-25T11:54:22.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.852+0000 7f3fe2ec2640 1 -- 172.21.15.83:0/253094064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fdc1a44b0 con 0x7f3fdc106170 2024-04-25T11:54:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.853+0000 7f3fe2ec2640 1 -- 172.21.15.83:0/253094064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fdc068d70 con 0x7f3fdc106170 2024-04-25T11:54:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.854+0000 7f3fd9ffb640 1 -- 172.21.15.83:0/253094064 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3fc8017070 con 0x7f3fdc106170 2024-04-25T11:54:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.855+0000 7f3fd9ffb640 1 --2- 172.21.15.83:0/253094064 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f3fcc0a3ec0 0x7f3fcc0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.855+0000 7f3fe0c5e640 1 --2- 172.21.15.83:0/253094064 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f3fcc0a3ec0 0x7f3fcc0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.855+0000 7f3fd9ffb640 1 -- 172.21.15.83:0/253094064 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f3fc80d0390 con 0x7f3fdc106170 2024-04-25T11:54:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.855+0000 7f3fe0c5e640 1 --2- 172.21.15.83:0/253094064 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f3fcc0a3ec0 0x7f3fcc0a6380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0031440 tx=0x7f3fd0005e80 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:22.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:22.859+0000 7f3fd9ffb640 1 -- 172.21.15.83:0/253094064 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3fc801e030 con 0x7f3fdc106170 2024-04-25T11:54:22.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:22 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3253243041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:54:22.907 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:22 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2152735809' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:54:23.002 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:22 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3253243041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:54:23.002 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:22 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2152735809' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:54:23.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:22 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3253243041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T11:54:23.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:22 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2152735809' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T11:54:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.073+0000 7f3fe2ec2640 1 -- 172.21.15.83:0/253094064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fdc101a50 con 0x7f3fdc106170 2024-04-25T11:54:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.074+0000 7f3fd9ffb640 1 -- 172.21.15.83:0/253094064 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11932 (secure 0 0 0) 0x7f3fc809ca30 con 0x7f3fdc106170 2024-04-25T11:54:23.063 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:54:23.063 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:41:44.787926+0000","modified":"2024-04-25T11:43:38.850704+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24469,"mds_2":24442,"mds_3":24440,"mds_4":24430},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/1568064835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1568064835},{"type":"v1","addr":"172.21.15.83:6839","nonce":1568064835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/1576544313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1576544313},{"type":"v1","addr":"172.21.15.83:6841","nonce":1576544313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":4,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.94:6837/2475976592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2475976592},{"type":"v1","addr":"172.21.15.94:6837","nonce":2475976592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.111:6837/2927430878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2927430878},{"type":"v1","addr":"172.21.15.111:6837","nonce":2927430878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/2233531151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2233531151},{"type":"v1","addr":"172.21.15.111:6839","nonce":2233531151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/1461078471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1461078471},{"type":"v1","addr":"172.21.15.94:6839","nonce":1461078471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24442,24469,24440,24479,24430]},"id":1}]} 2024-04-25T11:54:23.063 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2024-04-25T11:54:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.076+0000 7f3fe2ec2640 1 -- 172.21.15.83:0/253094064 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f3fcc0a3ec0 msgr2=0x7f3fcc0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.076+0000 7f3fe2ec2640 1 --2- 172.21.15.83:0/253094064 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f3fcc0a3ec0 0x7f3fcc0a6380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0031440 tx=0x7f3fd0005e80 comp rx=0 tx=0).stop 2024-04-25T11:54:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.076+0000 7f3fe2ec2640 1 -- 172.21.15.83:0/253094064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fdc106170 msgr2=0x7f3fdc197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.076+0000 7f3fe2ec2640 1 --2- 172.21.15.83:0/253094064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fdc106170 0x7f3fdc197bb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3fc800c970 tx=0x7f3fc800ce30 comp rx=0 tx=0).stop 2024-04-25T11:54:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.076+0000 7f3fe2ec2640 1 -- 172.21.15.83:0/253094064 shutdown_connections 2024-04-25T11:54:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.076+0000 7f3fe2ec2640 1 --2- 172.21.15.83:0/253094064 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f3fcc0a3ec0 0x7f3fcc0a6380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.076+0000 7f3fe2ec2640 1 --2- 172.21.15.83:0/253094064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fdc106170 0x7f3fdc197bb0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.076+0000 7f3fe2ec2640 1 --2- 172.21.15.83:0/253094064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fdc19bee0 0x7f3fdc19c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.076+0000 7f3fe2ec2640 1 --2- 172.21.15.83:0/253094064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3fdc1010a0 0x7f3fdc197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.076+0000 7f3fe2ec2640 1 -- 172.21.15.83:0/253094064 >> 172.21.15.83:0/253094064 conn(0x7f3fdc077f50 msgr2=0x7f3fdc0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.076+0000 7f3fe2ec2640 1 -- 172.21.15.83:0/253094064 shutdown_connections 2024-04-25T11:54:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.076+0000 7f3fe2ec2640 1 -- 172.21.15.83:0/253094064 wait complete. 2024-04-25T11:54:23.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:54:23.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.185+0000 7f4553908640 1 -- 172.21.15.83:0/2728962802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c101a40 msgr2=0x7f454c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:23.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.185+0000 7f4553908640 1 --2- 172.21.15.83:0/2728962802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c101a40 0x7f454c105a80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4548009900 tx=0x7f454802a410 comp rx=0 tx=0).stop 2024-04-25T11:54:23.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.185+0000 7f4553908640 1 -- 172.21.15.83:0/2728962802 shutdown_connections 2024-04-25T11:54:23.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.185+0000 7f4553908640 1 --2- 172.21.15.83:0/2728962802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c101a40 0x7f454c105a80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.185+0000 7f4553908640 1 --2- 172.21.15.83:0/2728962802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f454c106160 0x7f454c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.185+0000 7f4553908640 1 --2- 172.21.15.83:0/2728962802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f454c101090 0x7f454c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.185+0000 7f4553908640 1 -- 172.21.15.83:0/2728962802 >> 172.21.15.83:0/2728962802 conn(0x7f454c0771d0 msgr2=0x7f454c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:23.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.185+0000 7f4553908640 1 -- 172.21.15.83:0/2728962802 shutdown_connections 2024-04-25T11:54:23.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.185+0000 7f4553908640 1 -- 172.21.15.83:0/2728962802 wait complete. 2024-04-25T11:54:23.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4553908640 1 Processor -- start 2024-04-25T11:54:23.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4553908640 1 -- start start 2024-04-25T11:54:23.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4553908640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f454c101090 0x7f454c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:23.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4553908640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f454c106160 0x7f454c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:23.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4553908640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c19cf80 0x7f454c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:23.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4553908640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f454c1121a0 con 0x7f454c19cf80 2024-04-25T11:54:23.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4553908640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f454c112020 con 0x7f454c106160 2024-04-25T11:54:23.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4553908640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f454c112320 con 0x7f454c101090 2024-04-25T11:54:23.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4551ea5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c19cf80 0x7f454c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:23.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4551ea5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c19cf80 0x7f454c19d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37962/0 (socket says 172.21.15.83:37962) 2024-04-25T11:54:23.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4551ea5640 1 -- 172.21.15.83:0/4191484628 learned_addr learned my addr 172.21.15.83:0/4191484628 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:54:23.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f45516a4640 1 --2- 172.21.15.83:0/4191484628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f454c101090 0x7f454c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:23.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4550ea3640 1 --2- 172.21.15.83:0/4191484628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f454c106160 0x7f454c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:23.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4551ea5640 1 -- 172.21.15.83:0/4191484628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f454c101090 msgr2=0x7f454c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:23.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4551ea5640 1 --2- 172.21.15.83:0/4191484628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f454c101090 0x7f454c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4551ea5640 1 -- 172.21.15.83:0/4191484628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f454c106160 msgr2=0x7f454c198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:23.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4551ea5640 1 --2- 172.21.15.83:0/4191484628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f454c106160 0x7f454c198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f4551ea5640 1 -- 172.21.15.83:0/4191484628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454c19da70 con 0x7f454c19cf80 2024-04-25T11:54:23.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.186+0000 7f45516a4640 1 --2- 172.21.15.83:0/4191484628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f454c101090 0x7f454c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:54:23.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.187+0000 7f4551ea5640 1 --2- 172.21.15.83:0/4191484628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c19cf80 0x7f454c19d430 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f454400eaa0 tx=0x7f454400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:23.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.187+0000 7f45427fc640 1 -- 172.21.15.83:0/4191484628 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454400cdb0 con 0x7f454c19cf80 2024-04-25T11:54:23.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.187+0000 7f4553908640 1 -- 172.21.15.83:0/4191484628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454c19dd60 con 0x7f454c19cf80 2024-04-25T11:54:23.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.187+0000 7f45427fc640 1 -- 172.21.15.83:0/4191484628 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4544005020 con 0x7f454c19cf80 2024-04-25T11:54:23.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.187+0000 7f45427fc640 1 -- 172.21.15.83:0/4191484628 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45440116d0 con 0x7f454c19cf80 2024-04-25T11:54:23.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.187+0000 7f4553908640 1 -- 172.21.15.83:0/4191484628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f454c1a54d0 con 0x7f454c19cf80 2024-04-25T11:54:23.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.188+0000 7f4553908640 1 -- 172.21.15.83:0/4191484628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f454c0691c0 con 0x7f454c19cf80 2024-04-25T11:54:23.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.189+0000 7f45427fc640 1 -- 172.21.15.83:0/4191484628 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f45440051d0 con 0x7f454c19cf80 2024-04-25T11:54:23.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.189+0000 7f45427fc640 1 --2- 172.21.15.83:0/4191484628 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f45380a3d20 0x7f45380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:23.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.189+0000 7f45516a4640 1 --2- 172.21.15.83:0/4191484628 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f45380a3d20 0x7f45380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:23.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.189+0000 7f45427fc640 1 -- 172.21.15.83:0/4191484628 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f45440d09a0 con 0x7f454c19cf80 2024-04-25T11:54:23.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.190+0000 7f45516a4640 1 --2- 172.21.15.83:0/4191484628 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f45380a3d20 0x7f45380a61e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f453c00abe0 tx=0x7f453c00a5c0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:23.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.194+0000 7f45427fc640 1 -- 172.21.15.83:0/4191484628 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4544016030 con 0x7f454c19cf80 2024-04-25T11:54:23.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.410+0000 7f4553908640 1 -- 172.21.15.83:0/4191484628 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f454c101a40 con 0x7f454c19cf80 2024-04-25T11:54:23.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.410+0000 7f45427fc640 1 -- 172.21.15.83:0/4191484628 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11932 (secure 0 0 0) 0x7f454409d040 con 0x7f454c19cf80 2024-04-25T11:54:23.399 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:54:23.400 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3911246904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3911246904},{"type":"v1","addr":"172.21.15.83:6835","nonce":3911246904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3006959432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3006959432},{"type":"v1","addr":"172.21.15.83:6837","nonce":3006959432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T11:41:44.787926+0000","modified":"2024-04-25T11:43:38.850704+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24479,"mds_1":24469,"mds_2":24442,"mds_3":24440,"mds_4":24430},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6839/1568064835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":1568064835},{"type":"v1","addr":"172.21.15.83:6839","nonce":1568064835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6841/1576544313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":1576544313},{"type":"v1","addr":"172.21.15.83:6841","nonce":1576544313}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/2234401967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":2234401967},{"type":"v1","addr":"172.21.15.111:6833","nonce":2234401967}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24392":{"gid":24392,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6833/3250237836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3250237836},{"type":"v1","addr":"172.21.15.94:6833","nonce":3250237836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24416":{"gid":24416,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6835/4086957377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":4086957377},{"type":"v1","addr":"172.21.15.94:6835","nonce":4086957377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":4,"incarnation":23,"state":"up:active","state_seq":8,"addr":"172.21.15.111:6835/4167388778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4167388778},{"type":"v1","addr":"172.21.15.111:6835","nonce":4167388778}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"h","rank":3,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.94:6837/2475976592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2475976592},{"type":"v1","addr":"172.21.15.94:6837","nonce":2475976592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.111:6837/2927430878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":2927430878},{"type":"v1","addr":"172.21.15.111:6837","nonce":2927430878}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.111:6839/2233531151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2233531151},{"type":"v1","addr":"172.21.15.111:6839","nonce":2233531151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.94:6839/1461078471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":1461078471},{"type":"v1","addr":"172.21.15.94:6839","nonce":1461078471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24479,"qdb_cluster":[24442,24469,24440,24479,24430]},"id":1}]} 2024-04-25T11:54:23.400 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 25 2024-04-25T11:54:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.412+0000 7f4553908640 1 -- 172.21.15.83:0/4191484628 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f45380a3d20 msgr2=0x7f45380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:23.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.412+0000 7f4553908640 1 --2- 172.21.15.83:0/4191484628 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f45380a3d20 0x7f45380a61e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f453c00abe0 tx=0x7f453c00a5c0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.412+0000 7f4553908640 1 -- 172.21.15.83:0/4191484628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c19cf80 msgr2=0x7f454c19d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.412+0000 7f4553908640 1 --2- 172.21.15.83:0/4191484628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c19cf80 0x7f454c19d430 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f454400eaa0 tx=0x7f454400ef60 comp rx=0 tx=0).stop 2024-04-25T11:54:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.412+0000 7f4553908640 1 -- 172.21.15.83:0/4191484628 shutdown_connections 2024-04-25T11:54:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.412+0000 7f4553908640 1 --2- 172.21.15.83:0/4191484628 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f45380a3d20 0x7f45380a61e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.412+0000 7f4553908640 1 --2- 172.21.15.83:0/4191484628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f454c19cf80 0x7f454c19d430 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.412+0000 7f4553908640 1 --2- 172.21.15.83:0/4191484628 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f454c106160 0x7f454c198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.412+0000 7f4553908640 1 --2- 172.21.15.83:0/4191484628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f454c101090 0x7f454c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.412+0000 7f4553908640 1 -- 172.21.15.83:0/4191484628 >> 172.21.15.83:0/4191484628 conn(0x7f454c0771d0 msgr2=0x7f454c0fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.412+0000 7f4553908640 1 -- 172.21.15.83:0/4191484628 shutdown_connections 2024-04-25T11:54:23.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.413+0000 7f4553908640 1 -- 172.21.15.83:0/4191484628 wait complete. 2024-04-25T11:54:23.412 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:54:23.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.521+0000 7fa926d37640 1 -- 172.21.15.83:0/3686393872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9201010a0 msgr2=0x7fa920101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:23.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.521+0000 7fa926d37640 1 --2- 172.21.15.83:0/3686393872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9201010a0 0x7fa920101480 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa914009900 tx=0x7fa91402a410 comp rx=0 tx=0).stop 2024-04-25T11:54:23.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.521+0000 7fa926d37640 1 -- 172.21.15.83:0/3686393872 shutdown_connections 2024-04-25T11:54:23.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.521+0000 7fa926d37640 1 --2- 172.21.15.83:0/3686393872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa920106170 0x7fa92010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.521+0000 7fa926d37640 1 --2- 172.21.15.83:0/3686393872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa920101a50 0x7fa920105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.521+0000 7fa926d37640 1 --2- 172.21.15.83:0/3686393872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9201010a0 0x7fa920101480 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.521+0000 7fa926d37640 1 -- 172.21.15.83:0/3686393872 >> 172.21.15.83:0/3686393872 conn(0x7fa920077f50 msgr2=0x7fa920078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:23.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.522+0000 7fa926d37640 1 -- 172.21.15.83:0/3686393872 shutdown_connections 2024-04-25T11:54:23.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.522+0000 7fa926d37640 1 -- 172.21.15.83:0/3686393872 wait complete. 2024-04-25T11:54:23.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.522+0000 7fa926d37640 1 Processor -- start 2024-04-25T11:54:23.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.522+0000 7fa926d37640 1 -- start start 2024-04-25T11:54:23.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.522+0000 7fa926d37640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9201010a0 0x7fa92010e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:23.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.522+0000 7fa926d37640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa920106170 0x7fa92010ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:23.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.522+0000 7fa926d37640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa92010f150 0x7fa9201a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:23.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.522+0000 7fa926d37640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201121f0 con 0x7fa92010f150 2024-04-25T11:54:23.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.522+0000 7fa926d37640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa920112070 con 0x7fa920106170 2024-04-25T11:54:23.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.522+0000 7fa926d37640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa920112370 con 0x7fa9201010a0 2024-04-25T11:54:23.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.522+0000 7fa9252d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa92010f150 0x7fa9201a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:23.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.522+0000 7fa9252d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa92010f150 0x7fa9201a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37970/0 (socket says 172.21.15.83:37970) 2024-04-25T11:54:23.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.522+0000 7fa9252d4640 1 -- 172.21.15.83:0/1791651750 learned_addr learned my addr 172.21.15.83:0/1791651750 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:54:23.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.522+0000 7fa924ad3640 1 --2- 172.21.15.83:0/1791651750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9201010a0 0x7fa92010e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:23.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.523+0000 7fa91ffff640 1 --2- 172.21.15.83:0/1791651750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa920106170 0x7fa92010ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:23.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.523+0000 7fa9252d4640 1 -- 172.21.15.83:0/1791651750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9201010a0 msgr2=0x7fa92010e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.523+0000 7fa9252d4640 1 --2- 172.21.15.83:0/1791651750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9201010a0 0x7fa92010e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.523+0000 7fa9252d4640 1 -- 172.21.15.83:0/1791651750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa920106170 msgr2=0x7fa92010ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.523+0000 7fa9252d4640 1 --2- 172.21.15.83:0/1791651750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa920106170 0x7fa92010ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.523+0000 7fa9252d4640 1 -- 172.21.15.83:0/1791651750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9201a7020 con 0x7fa92010f150 2024-04-25T11:54:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.523+0000 7fa9252d4640 1 --2- 172.21.15.83:0/1791651750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa92010f150 0x7fa9201a6920 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa91800eaa0 tx=0x7fa91800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.523+0000 7fa91dffb640 1 -- 172.21.15.83:0/1791651750 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91800cdb0 con 0x7fa92010f150 2024-04-25T11:54:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.523+0000 7fa91dffb640 1 -- 172.21.15.83:0/1791651750 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa918005020 con 0x7fa92010f150 2024-04-25T11:54:23.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.523+0000 7fa926d37640 1 -- 172.21.15.83:0/1791651750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9201a7310 con 0x7fa92010f150 2024-04-25T11:54:23.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.523+0000 7fa91dffb640 1 -- 172.21.15.83:0/1791651750 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9180116d0 con 0x7fa92010f150 2024-04-25T11:54:23.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.523+0000 7fa926d37640 1 -- 172.21.15.83:0/1791651750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9201a7860 con 0x7fa92010f150 2024-04-25T11:54:23.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.524+0000 7fa926d37640 1 -- 172.21.15.83:0/1791651750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa920068d70 con 0x7fa92010f150 2024-04-25T11:54:23.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.525+0000 7fa91dffb640 1 -- 172.21.15.83:0/1791651750 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa9180051d0 con 0x7fa92010f150 2024-04-25T11:54:23.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.526+0000 7fa91dffb640 1 --2- 172.21.15.83:0/1791651750 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fa9100a3d50 0x7fa9100a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:23.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.526+0000 7fa924ad3640 1 --2- 172.21.15.83:0/1791651750 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fa9100a3d50 0x7fa9100a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:23.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.526+0000 7fa91dffb640 1 -- 172.21.15.83:0/1791651750 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7fa9180d0890 con 0x7fa92010f150 2024-04-25T11:54:23.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.526+0000 7fa924ad3640 1 --2- 172.21.15.83:0/1791651750 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fa9100a3d50 0x7fa9100a6210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa914031440 tx=0x7fa914005b50 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:23.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.531+0000 7fa91dffb640 1 -- 172.21.15.83:0/1791651750 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa918016030 con 0x7fa92010f150 2024-04-25T11:54:23.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.708+0000 7fa926d37640 1 -- 172.21.15.83:0/1791651750 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9201093a0 con 0x7fa92010f150 2024-04-25T11:54:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.709+0000 7fa91dffb640 1 -- 172.21.15.83:0/1791651750 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23048 (secure 0 0 0) 0x7fa918085e20 con 0x7fa92010f150 2024-04-25T11:54:23.698 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-25T11:54:23.699 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":99,"fsid":"7fcf54c6-02f7-11ef-bc93-c7b262605968","created":"2024-04-25T11:33:56.406187+0000","modified":"2024-04-25T11:54:16.743447+0000","last_up_change":"2024-04-25T11:40:57.961289+0000","last_in_change":"2024-04-25T11:40:38.435555+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:37:15.914779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:41:42.546991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:41:43.764226+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a79d318f-21ba-4f1b-a80c-54a8fa4618b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6803","nonce":2168030106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6805","nonce":2168030106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6809","nonce":2168030106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2168030106},{"type":"v1","addr":"172.21.15.83:6807","nonce":2168030106}]},"public_addr":"172.21.15.83:6803/2168030106","cluster_addr":"172.21.15.83:6805/2168030106","heartbeat_back_addr":"172.21.15.83:6809/2168030106","heartbeat_front_addr":"172.21.15.83:6807/2168030106","state":["exists","up"]},{"osd":1,"uuid":"e4540152-b87e-49a9-bf80-6de7cb165835","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6801","nonce":3582270380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6803","nonce":3582270380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6807","nonce":3582270380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3582270380},{"type":"v1","addr":"172.21.15.94:6805","nonce":3582270380}]},"public_addr":"172.21.15.94:6801/3582270380","cluster_addr":"172.21.15.94:6803/3582270380","heartbeat_back_addr":"172.21.15.94:6807/3582270380","heartbeat_front_addr":"172.21.15.94:6805/3582270380","state":["exists","up"]},{"osd":2,"uuid":"bdc6f455-7038-4178-b962-2de2d69180cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6801","nonce":101429764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6803","nonce":101429764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6807","nonce":101429764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":101429764},{"type":"v1","addr":"172.21.15.111:6805","nonce":101429764}]},"public_addr":"172.21.15.111:6801/101429764","cluster_addr":"172.21.15.111:6803/101429764","heartbeat_back_addr":"172.21.15.111:6807/101429764","heartbeat_front_addr":"172.21.15.111:6805/101429764","state":["exists","up"]},{"osd":3,"uuid":"1e927e50-a208-48ca-aeb4-1adc2ae48d55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6811","nonce":775479209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6813","nonce":775479209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6817","nonce":775479209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":775479209},{"type":"v1","addr":"172.21.15.83:6815","nonce":775479209}]},"public_addr":"172.21.15.83:6811/775479209","cluster_addr":"172.21.15.83:6813/775479209","heartbeat_back_addr":"172.21.15.83:6817/775479209","heartbeat_front_addr":"172.21.15.83:6815/775479209","state":["exists","up"]},{"osd":4,"uuid":"637e5b1b-12cb-41e9-9026-53faadfc28b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6809","nonce":768239394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6811","nonce":768239394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6815","nonce":768239394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":768239394},{"type":"v1","addr":"172.21.15.94:6813","nonce":768239394}]},"public_addr":"172.21.15.94:6809/768239394","cluster_addr":"172.21.15.94:6811/768239394","heartbeat_back_addr":"172.21.15.94:6815/768239394","heartbeat_front_addr":"172.21.15.94:6813/768239394","state":["exists","up"]},{"osd":5,"uuid":"050ab73c-e5d2-4140-8118-8f7c65b40229","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6809","nonce":3244351490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6811","nonce":3244351490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6815","nonce":3244351490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":3244351490},{"type":"v1","addr":"172.21.15.111:6813","nonce":3244351490}]},"public_addr":"172.21.15.111:6809/3244351490","cluster_addr":"172.21.15.111:6811/3244351490","heartbeat_back_addr":"172.21.15.111:6815/3244351490","heartbeat_front_addr":"172.21.15.111:6813/3244351490","state":["exists","up"]},{"osd":6,"uuid":"44e6b825-a33f-447c-bc4c-f1890016bd3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6819","nonce":305831773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6821","nonce":305831773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6825","nonce":305831773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":305831773},{"type":"v1","addr":"172.21.15.83:6823","nonce":305831773}]},"public_addr":"172.21.15.83:6819/305831773","cluster_addr":"172.21.15.83:6821/305831773","heartbeat_back_addr":"172.21.15.83:6825/305831773","heartbeat_front_addr":"172.21.15.83:6823/305831773","state":["exists","up"]},{"osd":7,"uuid":"50476b46-ec06-472f-bb91-920d81555099","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6817","nonce":4200260463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6819","nonce":4200260463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6823","nonce":4200260463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":4200260463},{"type":"v1","addr":"172.21.15.94:6821","nonce":4200260463}]},"public_addr":"172.21.15.94:6817/4200260463","cluster_addr":"172.21.15.94:6819/4200260463","heartbeat_back_addr":"172.21.15.94:6823/4200260463","heartbeat_front_addr":"172.21.15.94:6821/4200260463","state":["exists","up"]},{"osd":8,"uuid":"34ebcd68-bca8-4d62-b0df-728fd892a0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6817","nonce":297826032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6819","nonce":297826032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6823","nonce":297826032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":297826032},{"type":"v1","addr":"172.21.15.111:6821","nonce":297826032}]},"public_addr":"172.21.15.111:6817/297826032","cluster_addr":"172.21.15.111:6819/297826032","heartbeat_back_addr":"172.21.15.111:6823/297826032","heartbeat_front_addr":"172.21.15.111:6821/297826032","state":["exists","up"]},{"osd":9,"uuid":"9f70803a-8d30-40cd-8374-fefeb1bfc337","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6827","nonce":3768581100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6829","nonce":3768581100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6833","nonce":3768581100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3768581100},{"type":"v1","addr":"172.21.15.83:6831","nonce":3768581100}]},"public_addr":"172.21.15.83:6827/3768581100","cluster_addr":"172.21.15.83:6829/3768581100","heartbeat_back_addr":"172.21.15.83:6833/3768581100","heartbeat_front_addr":"172.21.15.83:6831/3768581100","state":["exists","up"]},{"osd":10,"uuid":"7d3d3f69-db95-47f3-aced-3eaa340839da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6825","nonce":2135498753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6827","nonce":2135498753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6831","nonce":2135498753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":2135498753},{"type":"v1","addr":"172.21.15.94:6829","nonce":2135498753}]},"public_addr":"172.21.15.94:6825/2135498753","cluster_addr":"172.21.15.94:6827/2135498753","heartbeat_back_addr":"172.21.15.94:6831/2135498753","heartbeat_front_addr":"172.21.15.94:6829/2135498753","state":["exists","up"]},{"osd":11,"uuid":"16ab0143-02e4-4a08-8cd3-3c9d9c841ad5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6825","nonce":4117294468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6827","nonce":4117294468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6831","nonce":4117294468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":4117294468},{"type":"v1","addr":"172.21.15.111:6829","nonce":4117294468}]},"public_addr":"172.21.15.111:6825/4117294468","cluster_addr":"172.21.15.111:6827/4117294468","heartbeat_back_addr":"172.21.15.111:6831/4117294468","heartbeat_front_addr":"172.21.15.111:6829/4117294468","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:25.299970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:36:49.957946+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:13.899796+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:37:38.575777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:02.971828+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:27.303709+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:38:51.885134+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:17.054181+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:39:41.456217+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:06.443369+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:31.298146+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:40:56.288264+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:6801/1157946519":"2024-04-26T11:43:40.200248+0000","172.21.15.83:0/3934887058":"2024-04-26T11:43:40.200248+0000","172.21.15.83:0/457699907":"2024-04-26T11:43:40.200248+0000","172.21.15.83:0/486441637":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/1837678414":"2024-04-26T11:43:40.200248+0000","172.21.15.83:0/1048355479":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6801/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/1639787889":"2024-04-26T11:34:49.826109+0000","172.21.15.83:0/3257452011":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/908199980":"2024-04-26T11:34:49.826109+0000","172.21.15.83:6800/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/2677315213":"2024-04-26T11:34:23.164299+0000","172.21.15.83:6800/1157946519":"2024-04-26T11:43:40.200248+0000","172.21.15.83:0/3642996010":"2024-04-26T11:43:40.200248+0000","172.21.15.83:6801/3997740373":"2024-04-26T11:34:23.164299+0000","172.21.15.83:0/1174617340":"2024-04-26T11:34:23.164299+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:54:23.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.712+0000 7fa926d37640 1 -- 172.21.15.83:0/1791651750 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fa9100a3d50 msgr2=0x7fa9100a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.712+0000 7fa926d37640 1 --2- 172.21.15.83:0/1791651750 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fa9100a3d50 0x7fa9100a6210 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa914031440 tx=0x7fa914005b50 comp rx=0 tx=0).stop 2024-04-25T11:54:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.712+0000 7fa926d37640 1 -- 172.21.15.83:0/1791651750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa92010f150 msgr2=0x7fa9201a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.712+0000 7fa926d37640 1 --2- 172.21.15.83:0/1791651750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa92010f150 0x7fa9201a6920 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa91800eaa0 tx=0x7fa91800ef60 comp rx=0 tx=0).stop 2024-04-25T11:54:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.712+0000 7fa926d37640 1 -- 172.21.15.83:0/1791651750 shutdown_connections 2024-04-25T11:54:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.712+0000 7fa926d37640 1 --2- 172.21.15.83:0/1791651750 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7fa9100a3d50 0x7fa9100a6210 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.712+0000 7fa926d37640 1 --2- 172.21.15.83:0/1791651750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa92010f150 0x7fa9201a6920 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.712+0000 7fa926d37640 1 --2- 172.21.15.83:0/1791651750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa920106170 0x7fa92010ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.712+0000 7fa926d37640 1 --2- 172.21.15.83:0/1791651750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9201010a0 0x7fa92010e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.712+0000 7fa926d37640 1 -- 172.21.15.83:0/1791651750 >> 172.21.15.83:0/1791651750 conn(0x7fa920077f50 msgr2=0x7fa9200ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:23.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.712+0000 7fa926d37640 1 -- 172.21.15.83:0/1791651750 shutdown_connections 2024-04-25T11:54:23.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.712+0000 7fa926d37640 1 -- 172.21.15.83:0/1791651750 wait complete. 2024-04-25T11:54:23.712 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:54:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.822+0000 7f2c954c2640 1 -- 172.21.15.83:0/1190170435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c90102b60 msgr2=0x7f2c90106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.822+0000 7f2c954c2640 1 --2- 172.21.15.83:0/1190170435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c90102b60 0x7f2c90106ba0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009900 tx=0x7f2c8402a410 comp rx=0 tx=0).stop 2024-04-25T11:54:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.822+0000 7f2c954c2640 1 -- 172.21.15.83:0/1190170435 shutdown_connections 2024-04-25T11:54:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.822+0000 7f2c954c2640 1 --2- 172.21.15.83:0/1190170435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c90102b60 0x7f2c90106ba0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.822+0000 7f2c954c2640 1 --2- 172.21.15.83:0/1190170435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c90107280 0x7f2c9010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.822+0000 7f2c954c2640 1 --2- 172.21.15.83:0/1190170435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c901021b0 0x7f2c90102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.822+0000 7f2c954c2640 1 -- 172.21.15.83:0/1190170435 >> 172.21.15.83:0/1190170435 conn(0x7f2c900fd9f0 msgr2=0x7f2c900ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.822+0000 7f2c954c2640 1 -- 172.21.15.83:0/1190170435 shutdown_connections 2024-04-25T11:54:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.822+0000 7f2c954c2640 1 -- 172.21.15.83:0/1190170435 wait complete. 2024-04-25T11:54:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.823+0000 7f2c954c2640 1 Processor -- start 2024-04-25T11:54:23.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.823+0000 7f2c954c2640 1 -- start start 2024-04-25T11:54:23.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.823+0000 7f2c954c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c901021b0 0x7f2c9019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:23.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.823+0000 7f2c954c2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c90107280 0x7f2c9019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:23.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.823+0000 7f2c954c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c90198640 0x7f2c90198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:23.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.823+0000 7f2c954c2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c901121c0 con 0x7f2c901021b0 2024-04-25T11:54:23.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.823+0000 7f2c954c2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90112040 con 0x7f2c90198640 2024-04-25T11:54:23.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.823+0000 7f2c954c2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2c90112340 con 0x7f2c90107280 2024-04-25T11:54:23.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.823+0000 7f2c8effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c901021b0 0x7f2c9019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:23.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.823+0000 7f2c8effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c901021b0 0x7f2c9019e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37978/0 (socket says 172.21.15.83:37978) 2024-04-25T11:54:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.823+0000 7f2c8effd640 1 -- 172.21.15.83:0/391504882 learned_addr learned my addr 172.21.15.83:0/391504882 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:54:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.823+0000 7f2c8e7fc640 1 --2- 172.21.15.83:0/391504882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c90107280 0x7f2c9019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.824+0000 7f2c8f7fe640 1 --2- 172.21.15.83:0/391504882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c90198640 0x7f2c90198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.824+0000 7f2c8effd640 1 -- 172.21.15.83:0/391504882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c90107280 msgr2=0x7f2c9019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.824+0000 7f2c8effd640 1 --2- 172.21.15.83:0/391504882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c90107280 0x7f2c9019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.824+0000 7f2c8effd640 1 -- 172.21.15.83:0/391504882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c90198640 msgr2=0x7f2c90198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.824+0000 7f2c8effd640 1 --2- 172.21.15.83:0/391504882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c90198640 0x7f2c90198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.824+0000 7f2c8effd640 1 -- 172.21.15.83:0/391504882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c90199240 con 0x7f2c901021b0 2024-04-25T11:54:23.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.824+0000 7f2c8f7fe640 1 --2- 172.21.15.83:0/391504882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c90198640 0x7f2c90198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:54:23.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.824+0000 7f2c8effd640 1 --2- 172.21.15.83:0/391504882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c901021b0 0x7f2c9019e580 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000c890 tx=0x7f2c8000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:23.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.824+0000 7f2c77fff640 1 -- 172.21.15.83:0/391504882 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80004220 con 0x7f2c901021b0 2024-04-25T11:54:23.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.824+0000 7f2c77fff640 1 -- 172.21.15.83:0/391504882 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c800043c0 con 0x7f2c901021b0 2024-04-25T11:54:23.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.824+0000 7f2c954c2640 1 -- 172.21.15.83:0/391504882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c901a5320 con 0x7f2c901021b0 2024-04-25T11:54:23.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.824+0000 7f2c77fff640 1 -- 172.21.15.83:0/391504882 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80004a00 con 0x7f2c901021b0 2024-04-25T11:54:23.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.824+0000 7f2c954c2640 1 -- 172.21.15.83:0/391504882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c901a56d0 con 0x7f2c901021b0 2024-04-25T11:54:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.825+0000 7f2c954c2640 1 -- 172.21.15.83:0/391504882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c90103680 con 0x7f2c901021b0 2024-04-25T11:54:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.826+0000 7f2c77fff640 1 -- 172.21.15.83:0/391504882 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c800079b0 con 0x7f2c901021b0 2024-04-25T11:54:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.826+0000 7f2c77fff640 1 --2- 172.21.15.83:0/391504882 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f2c780a3d20 0x7f2c780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:23.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.827+0000 7f2c8e7fc640 1 --2- 172.21.15.83:0/391504882 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f2c780a3d20 0x7f2c780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:23.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.827+0000 7f2c77fff640 1 -- 172.21.15.83:0/391504882 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8513+0+0 (secure 0 0 0) 0x7f2c800d0550 con 0x7f2c901021b0 2024-04-25T11:54:23.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.827+0000 7f2c8e7fc640 1 --2- 172.21.15.83:0/391504882 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f2c780a3d20 0x7f2c780a61e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c84000c00 tx=0x7f2c84031400 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:23.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:23.831+0000 7f2c77fff640 1 -- 172.21.15.83:0/391504882 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c80014030 con 0x7f2c901021b0 2024-04-25T11:54:24.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:23 smithi094 ceph-mon[33032]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:54:24.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:23 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/253094064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:54:24.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:23 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/4191484628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:54:24.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:23 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1791651750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:54:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.044+0000 7f2c954c2640 1 -- 172.21.15.83:0/391504882 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2c9010a4b0 con 0x7f2c901021b0 2024-04-25T11:54:24.034 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:23 smithi083 ceph-mon[29330]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:54:24.035 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:23 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/253094064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:54:24.035 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:23 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/4191484628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:54:24.035 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:23 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1791651750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:54:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.046+0000 7f2c77fff640 1 -- 172.21.15.83:0/391504882 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7f2c8009cbf0 con 0x7f2c901021b0 2024-04-25T11:54:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:54:24.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.050+0000 7f2c954c2640 1 -- 172.21.15.83:0/391504882 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f2c780a3d20 msgr2=0x7f2c780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:24.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.050+0000 7f2c954c2640 1 --2- 172.21.15.83:0/391504882 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f2c780a3d20 0x7f2c780a61e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2c84000c00 tx=0x7f2c84031400 comp rx=0 tx=0).stop 2024-04-25T11:54:24.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.050+0000 7f2c954c2640 1 -- 172.21.15.83:0/391504882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c901021b0 msgr2=0x7f2c9019e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.050+0000 7f2c954c2640 1 --2- 172.21.15.83:0/391504882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c901021b0 0x7f2c9019e580 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000c890 tx=0x7f2c8000cd50 comp rx=0 tx=0).stop 2024-04-25T11:54:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.050+0000 7f2c954c2640 1 -- 172.21.15.83:0/391504882 shutdown_connections 2024-04-25T11:54:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.050+0000 7f2c954c2640 1 --2- 172.21.15.83:0/391504882 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f2c780a3d20 0x7f2c780a61e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.050+0000 7f2c954c2640 1 --2- 172.21.15.83:0/391504882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c90198640 0x7f2c90198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.050+0000 7f2c954c2640 1 --2- 172.21.15.83:0/391504882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c90107280 0x7f2c9019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.050+0000 7f2c954c2640 1 --2- 172.21.15.83:0/391504882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c901021b0 0x7f2c9019e580 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.050+0000 7f2c954c2640 1 -- 172.21.15.83:0/391504882 >> 172.21.15.83:0/391504882 conn(0x7f2c900fd9f0 msgr2=0x7f2c900fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.050+0000 7f2c954c2640 1 -- 172.21.15.83:0/391504882 shutdown_connections 2024-04-25T11:54:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.050+0000 7f2c954c2640 1 -- 172.21.15.83:0/391504882 wait complete. 2024-04-25T11:54:24.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:54:24.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:23 smithi111 ceph-mon[32784]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-25T11:54:24.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:23 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/253094064' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:54:24.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:23 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/4191484628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T11:54:24.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:23 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1791651750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T11:54:24.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.167+0000 7efed5d07640 1 -- 172.21.15.83:0/1882849192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed01031d0 msgr2=0x7efed01035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:24.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.167+0000 7efed5d07640 1 --2- 172.21.15.83:0/1882849192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed01031d0 0x7efed01035b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7efec0009900 tx=0x7efec002a510 comp rx=0 tx=0).stop 2024-04-25T11:54:24.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.168+0000 7efed5d07640 1 -- 172.21.15.83:0/1882849192 shutdown_connections 2024-04-25T11:54:24.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.168+0000 7efed5d07640 1 --2- 172.21.15.83:0/1882849192 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed010c510 0x7efed010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:24.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.168+0000 7efed5d07640 1 --2- 172.21.15.83:0/1882849192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efed0103af0 0x7efed010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:24.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.168+0000 7efed5d07640 1 --2- 172.21.15.83:0/1882849192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed01031d0 0x7efed01035b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:24.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.168+0000 7efed5d07640 1 -- 172.21.15.83:0/1882849192 >> 172.21.15.83:0/1882849192 conn(0x7efed0077f50 msgr2=0x7efed0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:24.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.168+0000 7efed5d07640 1 -- 172.21.15.83:0/1882849192 shutdown_connections 2024-04-25T11:54:24.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.168+0000 7efed5d07640 1 -- 172.21.15.83:0/1882849192 wait complete. 2024-04-25T11:54:24.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.168+0000 7efed5d07640 1 Processor -- start 2024-04-25T11:54:24.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.168+0000 7efed5d07640 1 -- start start 2024-04-25T11:54:24.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.168+0000 7efed5d07640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed01031d0 0x7efed0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:24.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.168+0000 7efed5d07640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efed0103af0 0x7efed0198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:24.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efed5d07640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed010c510 0x7efed019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:24.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efed5d07640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efed0112180 con 0x7efed01031d0 2024-04-25T11:54:24.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efed5d07640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efed0112000 con 0x7efed010c510 2024-04-25T11:54:24.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efed5d07640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efed0112300 con 0x7efed0103af0 2024-04-25T11:54:24.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efecf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed01031d0 0x7efed0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:24.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efecf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed01031d0 0x7efed0198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38016/0 (socket says 172.21.15.83:38016) 2024-04-25T11:54:24.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efecf7fe640 1 -- 172.21.15.83:0/3934589586 learned_addr learned my addr 172.21.15.83:0/3934589586 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:54:24.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efecffff640 1 --2- 172.21.15.83:0/3934589586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed010c510 0x7efed019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:24.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efec6ffd640 1 --2- 172.21.15.83:0/3934589586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efed0103af0 0x7efed0198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:24.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efecf7fe640 1 -- 172.21.15.83:0/3934589586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efed0103af0 msgr2=0x7efed0198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:24.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efecf7fe640 1 --2- 172.21.15.83:0/3934589586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efed0103af0 0x7efed0198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:24.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efecf7fe640 1 -- 172.21.15.83:0/3934589586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed010c510 msgr2=0x7efed019d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:24.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efecf7fe640 1 --2- 172.21.15.83:0/3934589586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed010c510 0x7efed019d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:24.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efecf7fe640 1 -- 172.21.15.83:0/3934589586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed019d730 con 0x7efed01031d0 2024-04-25T11:54:24.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efec6ffd640 1 --2- 172.21.15.83:0/3934589586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efed0103af0 0x7efed0198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:54:24.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efecf7fe640 1 --2- 172.21.15.83:0/3934589586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed01031d0 0x7efed0198740 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efec0005940 tx=0x7efec0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:24.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efecd7fa640 1 -- 172.21.15.83:0/3934589586 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec003f030 con 0x7efed01031d0 2024-04-25T11:54:24.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efecd7fa640 1 -- 172.21.15.83:0/3934589586 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec0003aa0 con 0x7efed01031d0 2024-04-25T11:54:24.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efed5d07640 1 -- 172.21.15.83:0/3934589586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed019d9c0 con 0x7efed01031d0 2024-04-25T11:54:24.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efecd7fa640 1 -- 172.21.15.83:0/3934589586 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0035070 con 0x7efed01031d0 2024-04-25T11:54:24.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.169+0000 7efed5d07640 1 -- 172.21.15.83:0/3934589586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed01a52b0 con 0x7efed01031d0 2024-04-25T11:54:24.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.170+0000 7efed5d07640 1 -- 172.21.15.83:0/3934589586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed0068d70 con 0x7efed01031d0 2024-04-25T11:54:24.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.171+0000 7efecd7fa640 1 -- 172.21.15.83:0/3934589586 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efec0042030 con 0x7efed01031d0 2024-04-25T11:54:24.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.172+0000 7efecd7fa640 1 --2- 172.21.15.83:0/3934589586 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7efebc0a3d20 0x7efebc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:24.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.172+0000 7efec6ffd640 1 --2- 172.21.15.83:0/3934589586 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7efebc0a3d20 0x7efebc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:24.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.172+0000 7efecd7fa640 1 -- 172.21.15.83:0/3934589586 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8899+0+0 (secure 0 0 0) 0x7efec00f1260 con 0x7efed01031d0 2024-04-25T11:54:24.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.173+0000 7efec6ffd640 1 --2- 172.21.15.83:0/3934589586 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7efebc0a3d20 0x7efebc0a61e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efeb8005d00 tx=0x7efeb8009550 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:24.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.176+0000 7efecd7fa640 1 -- 172.21.15.83:0/3934589586 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efec00bd720 con 0x7efed01031d0 2024-04-25T11:54:24.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:24.394+0000 7efed5d07640 1 -- 172.21.15.83:0/3934589586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7efed0107690 con 0x7efed01031d0 2024-04-25T11:54:24.407 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a[29326]: 2024-04-25T11:54:24.045+0000 7f331fd50700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:54:24.408 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-a[71742]: ignoring --setuser ceph since I am not root 2024-04-25T11:54:24.408 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-a[71742]: ignoring --setgroup ceph since I am not root 2024-04-25T11:54:24.408 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-mds[71746]: -- 172.21.15.83:0/355638634 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5609ff575a40 con 0x5609ff548c00 2024-04-25T11:54:24.408 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-a[71742]: starting mds.a at 2024-04-25T11:54:24.408 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-g[71378]: ignoring --setuser ceph since I am not root 2024-04-25T11:54:24.408 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-g[71378]: ignoring --setgroup ceph since I am not root 2024-04-25T11:54:24.409 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-mds[71382]: -- 172.21.15.83:0/3636526586 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b9272dba40 con 0x55b9272ae800 2024-04-25T11:54:24.409 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-g[71378]: starting mds.g at 2024-04-25T11:54:24.524 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-e[62653]: ignoring --setuser ceph since I am not root 2024-04-25T11:54:24.525 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-e[62653]: ignoring --setgroup ceph since I am not root 2024-04-25T11:54:24.525 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-mds[62657]: -- 172.21.15.94:0/1711506292 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5604559e5a40 con 0x5604559b8800 2024-04-25T11:54:24.525 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-e[62653]: starting mds.e at 2024-04-25T11:54:24.525 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b[63285]: ignoring --setuser ceph since I am not root 2024-04-25T11:54:24.525 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b[63285]: ignoring --setgroup ceph since I am not root 2024-04-25T11:54:24.526 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-mds[63289]: -- 172.21.15.94:0/2101186316 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5572086ffa40 con 0x5572086d2c00 2024-04-25T11:54:24.526 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b[63285]: starting mds.b at 2024-04-25T11:54:24.526 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-h[62961]: ignoring --setuser ceph since I am not root 2024-04-25T11:54:24.526 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-h[62961]: ignoring --setgroup ceph since I am not root 2024-04-25T11:54:24.526 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-mds[62965]: -- 172.21.15.94:0/3584708736 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f74178fa40 con 0x55f741762c00 2024-04-25T11:54:24.526 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-h[62961]: starting mds.h at 2024-04-25T11:54:24.527 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-k[62329]: ignoring --setuser ceph since I am not root 2024-04-25T11:54:24.527 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-k[62329]: ignoring --setgroup ceph since I am not root 2024-04-25T11:54:24.527 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-mds[62333]: -- 172.21.15.94:0/65228457 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5558169ada40 con 0x555816980800 2024-04-25T11:54:24.527 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-k[62329]: starting mds.k at 2024-04-25T11:54:24.573 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-f[63191]: ignoring --setuser ceph since I am not root 2024-04-25T11:54:24.573 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-f[63191]: ignoring --setgroup ceph since I am not root 2024-04-25T11:54:24.574 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-f[63191]: starting mds.f at 2024-04-25T11:54:24.574 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-c[63499]: ignoring --setuser ceph since I am not root 2024-04-25T11:54:24.574 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-c[63499]: ignoring --setgroup ceph since I am not root 2024-04-25T11:54:24.574 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-mds[63503]: -- 172.21.15.111:0/188359745 <== mon.2 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad4c5a9a40 con 0x55ad4c57cc00 2024-04-25T11:54:24.575 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-c[63499]: starting mds.c at 2024-04-25T11:54:24.575 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-l[62887]: ignoring --setuser ceph since I am not root 2024-04-25T11:54:24.575 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-l[62887]: ignoring --setgroup ceph since I am not root 2024-04-25T11:54:24.575 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-mds[62891]: -- 172.21.15.111:0/3842663747 <== mon.1 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559632c9fa40 con 0x559632c72c00 2024-04-25T11:54:24.575 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-l[62887]: starting mds.l at 2024-04-25T11:54:24.576 INFO:journalctl@ceph.mds.i.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-i[63813]: ignoring --setuser ceph since I am not root 2024-04-25T11:54:24.576 INFO:journalctl@ceph.mds.i.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-i[63813]: ignoring --setgroup ceph since I am not root 2024-04-25T11:54:24.576 INFO:journalctl@ceph.mds.i.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-mds[63817]: -- 172.21.15.111:0/3377403137 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a0e8ca9a40 con 0x55a0e8c7c800 2024-04-25T11:54:24.576 INFO:journalctl@ceph.mds.i.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-i[63813]: starting mds.i at 2024-04-25T11:54:25.024 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/391504882' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T11:54:25.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-mon[33032]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T11:54:25.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-mon[33032]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:54:25.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/391504882' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T11:54:25.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-mon[33032]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T11:54:25.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-mon[33032]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T11:54:25.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:24 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3934589586' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:54:25.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.051+0000 7efecd7fa640 1 -- 172.21.15.83:0/3934589586 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7efec00bd020 con 0x7efed01031d0 2024-04-25T11:54:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.054+0000 7efed5d07640 1 -- 172.21.15.83:0/3934589586 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7efebc0a3d20 msgr2=0x7efebc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.054+0000 7efed5d07640 1 --2- 172.21.15.83:0/3934589586 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7efebc0a3d20 0x7efebc0a61e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efeb8005d00 tx=0x7efeb8009550 comp rx=0 tx=0).stop 2024-04-25T11:54:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.054+0000 7efed5d07640 1 -- 172.21.15.83:0/3934589586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed01031d0 msgr2=0x7efed0198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.054+0000 7efed5d07640 1 --2- 172.21.15.83:0/3934589586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed01031d0 0x7efed0198740 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efec0005940 tx=0x7efec0004270 comp rx=0 tx=0).stop 2024-04-25T11:54:25.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.054+0000 7efed5d07640 1 -- 172.21.15.83:0/3934589586 shutdown_connections 2024-04-25T11:54:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.054+0000 7efed5d07640 1 --2- 172.21.15.83:0/3934589586 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7efebc0a3d20 0x7efebc0a61e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.054+0000 7efed5d07640 1 --2- 172.21.15.83:0/3934589586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efed010c510 0x7efed019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.054+0000 7efed5d07640 1 --2- 172.21.15.83:0/3934589586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efed0103af0 0x7efed0198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.054+0000 7efed5d07640 1 --2- 172.21.15.83:0/3934589586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efed01031d0 0x7efed0198740 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.054+0000 7efed5d07640 1 -- 172.21.15.83:0/3934589586 >> 172.21.15.83:0/3934589586 conn(0x7efed0077f50 msgr2=0x7efed00fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.054+0000 7efed5d07640 1 -- 172.21.15.83:0/3934589586 shutdown_connections 2024-04-25T11:54:25.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.054+0000 7efed5d07640 1 -- 172.21.15.83:0/3934589586 wait complete. 2024-04-25T11:54:25.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/391504882' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T11:54:25.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-mon[29330]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T11:54:25.059 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-mon[29330]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:54:25.059 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/391504882' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T11:54:25.059 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-mon[29330]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T11:54:25.059 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-mon[29330]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T11:54:25.059 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:24 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3934589586' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:54:25.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:54:25.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/391504882' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T11:54:25.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-mon[32784]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T11:54:25.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-mon[32784]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:54:25.073 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/391504882' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T11:54:25.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-mon[32784]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T11:54:25.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-mon[32784]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-25T11:54:25.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:24 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3934589586' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T11:54:25.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.177+0000 7f71df6fa640 1 -- 172.21.15.83:0/3750079944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d81018e0 msgr2=0x7f71d8105da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:25.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.177+0000 7f71df6fa640 1 --2- 172.21.15.83:0/3750079944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d81018e0 0x7f71d8105da0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f71c8009900 tx=0x7f71c802a510 comp rx=0 tx=0).stop 2024-04-25T11:54:25.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.177+0000 7f71df6fa640 1 -- 172.21.15.83:0/3750079944 shutdown_connections 2024-04-25T11:54:25.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.177+0000 7f71df6fa640 1 --2- 172.21.15.83:0/3750079944 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d81018e0 0x7f71d8105da0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:25.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.177+0000 7f71df6fa640 1 --2- 172.21.15.83:0/3750079944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71d81062e0 0x7f71d8106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:25.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.177+0000 7f71df6fa640 1 --2- 172.21.15.83:0/3750079944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71d8100fc0 0x7f71d81013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:25.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.177+0000 7f71df6fa640 1 -- 172.21.15.83:0/3750079944 >> 172.21.15.83:0/3750079944 conn(0x7f71d8077f50 msgr2=0x7f71d8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:25.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.177+0000 7f71df6fa640 1 -- 172.21.15.83:0/3750079944 shutdown_connections 2024-04-25T11:54:25.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.177+0000 7f71df6fa640 1 -- 172.21.15.83:0/3750079944 wait complete. 2024-04-25T11:54:25.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71df6fa640 1 Processor -- start 2024-04-25T11:54:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71df6fa640 1 -- start start 2024-04-25T11:54:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71df6fa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71d8100fc0 0x7f71d819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71df6fa640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71d81018e0 0x7f71d819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:25.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71df6fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d81062e0 0x7f71d81a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:25.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71df6fa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71d81187a0 con 0x7f71d81062e0 2024-04-25T11:54:25.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71df6fa640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8118620 con 0x7f71d8100fc0 2024-04-25T11:54:25.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71df6fa640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8118920 con 0x7f71d81018e0 2024-04-25T11:54:25.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71ddc97640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d81062e0 0x7f71d81a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:25.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71ddc97640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d81062e0 0x7f71d81a36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38024/0 (socket says 172.21.15.83:38024) 2024-04-25T11:54:25.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71ddc97640 1 -- 172.21.15.83:0/2788313351 learned_addr learned my addr 172.21.15.83:0/2788313351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:54:25.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71dd496640 1 --2- 172.21.15.83:0/2788313351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71d8100fc0 0x7f71d819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:25.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71dcc95640 1 --2- 172.21.15.83:0/2788313351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71d81018e0 0x7f71d819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:25.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71ddc97640 1 -- 172.21.15.83:0/2788313351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71d81018e0 msgr2=0x7f71d819f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:25.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71ddc97640 1 --2- 172.21.15.83:0/2788313351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71d81018e0 0x7f71d819f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:25.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71ddc97640 1 -- 172.21.15.83:0/2788313351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71d8100fc0 msgr2=0x7f71d819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:25.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71ddc97640 1 --2- 172.21.15.83:0/2788313351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71d8100fc0 0x7f71d819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:25.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.178+0000 7f71ddc97640 1 -- 172.21.15.83:0/2788313351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d81a3dd0 con 0x7f71d81062e0 2024-04-25T11:54:25.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.179+0000 7f71ddc97640 1 --2- 172.21.15.83:0/2788313351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d81062e0 0x7f71d81a36a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f71d400eaa0 tx=0x7f71d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:25.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.179+0000 7f71ce7fc640 1 -- 172.21.15.83:0/2788313351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d400cdb0 con 0x7f71d81062e0 2024-04-25T11:54:25.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.179+0000 7f71ce7fc640 1 -- 172.21.15.83:0/2788313351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d4005020 con 0x7f71d81062e0 2024-04-25T11:54:25.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.179+0000 7f71df6fa640 1 -- 172.21.15.83:0/2788313351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d81a40c0 con 0x7f71d81062e0 2024-04-25T11:54:25.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.179+0000 7f71ce7fc640 1 -- 172.21.15.83:0/2788313351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d40116d0 con 0x7f71d81062e0 2024-04-25T11:54:25.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.179+0000 7f71df6fa640 1 -- 172.21.15.83:0/2788313351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d81ab950 con 0x7f71d81062e0 2024-04-25T11:54:25.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.180+0000 7f71df6fa640 1 -- 172.21.15.83:0/2788313351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d8068d70 con 0x7f71d81062e0 2024-04-25T11:54:25.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.181+0000 7f71ce7fc640 1 -- 172.21.15.83:0/2788313351 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f71d40051d0 con 0x7f71d81062e0 2024-04-25T11:54:25.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.181+0000 7f71ce7fc640 1 --2- 172.21.15.83:0/2788313351 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f71c40a3d20 0x7f71c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:25.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.182+0000 7f71dd496640 1 --2- 172.21.15.83:0/2788313351 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f71c40a3d20 0x7f71c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:25.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.182+0000 7f71ce7fc640 1 -- 172.21.15.83:0/2788313351 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8899+0+0 (secure 0 0 0) 0x7f71d40d0a40 con 0x7f71d81062e0 2024-04-25T11:54:25.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.182+0000 7f71dd496640 1 --2- 172.21.15.83:0/2788313351 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f71c40a3d20 0x7f71c40a61e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f71d000bbe0 tx=0x7f71d000b390 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:25.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.186+0000 7f71ce7fc640 1 -- 172.21.15.83:0/2788313351 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f71d4016030 con 0x7f71d81062e0 2024-04-25T11:54:25.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:25.362+0000 7f71df6fa640 1 -- 172.21.15.83:0/2788313351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f71d81abbb0 con 0x7f71d81062e0 2024-04-25T11:54:26.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-25T11:54:26.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T11:54:26.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T11:54:26.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: Cluster is now healthy 2024-04-25T11:54:26.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/3934589586' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T11:54:26.025 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.83:6838/325778165,v1:172.21.15.83:6841/325778165] up:boot 2024-04-25T11:54:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.83:6839/1778381896,v1:172.21.15.83:6840/1778381896] up:boot 2024-04-25T11:54:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.94:6833/1576549222,v1:172.21.15.94:6835/1576549222] up:boot 2024-04-25T11:54:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.94:6832/325114104,v1:172.21.15.94:6834/325114104] up:boot 2024-04-25T11:54:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.94:6836/3617346119,v1:172.21.15.94:6837/3617346119] up:boot 2024-04-25T11:54:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.94:6838/3681861560,v1:172.21.15.94:6839/3681861560] up:boot 2024-04-25T11:54:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.111:6832/703193743,v1:172.21.15.111:6834/703193743] up:boot 2024-04-25T11:54:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.111:6833/2681315840,v1:172.21.15.111:6835/2681315840] up:boot 2024-04-25T11:54:26.026 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.111:6836/910979687,v1:172.21.15.111:6838/910979687] up:boot 2024-04-25T11:54:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: mds.? [v2:172.21.15.111:6837/4041192626,v1:172.21.15.111:6839/4041192626] up:boot 2024-04-25T11:54:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: fsmap 12 up:standby 2024-04-25T11:54:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T11:54:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T11:54:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T11:54:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:54:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T11:54:26.027 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:54:26.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T11:54:26.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T11:54:26.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T11:54:26.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T11:54:26.028 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:25 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2788313351' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:54:26.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.053+0000 7f71ce7fc640 1 -- 172.21.15.83:0/2788313351 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) ==== 181+0+0 (secure 0 0 0) 0x7f71d409c910 con 0x7f71d81062e0 2024-04-25T11:54:26.043 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:54:26.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.056+0000 7f71df6fa640 1 -- 172.21.15.83:0/2788313351 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f71c40a3d20 msgr2=0x7f71c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:26.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.056+0000 7f71df6fa640 1 --2- 172.21.15.83:0/2788313351 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f71c40a3d20 0x7f71c40a61e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f71d000bbe0 tx=0x7f71d000b390 comp rx=0 tx=0).stop 2024-04-25T11:54:26.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.056+0000 7f71df6fa640 1 -- 172.21.15.83:0/2788313351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d81062e0 msgr2=0x7f71d81a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:26.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.056+0000 7f71df6fa640 1 --2- 172.21.15.83:0/2788313351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d81062e0 0x7f71d81a36a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f71d400eaa0 tx=0x7f71d400ef60 comp rx=0 tx=0).stop 2024-04-25T11:54:26.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.056+0000 7f71df6fa640 1 -- 172.21.15.83:0/2788313351 shutdown_connections 2024-04-25T11:54:26.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.056+0000 7f71df6fa640 1 --2- 172.21.15.83:0/2788313351 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f71c40a3d20 0x7f71c40a61e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:26.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.056+0000 7f71df6fa640 1 --2- 172.21.15.83:0/2788313351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d81062e0 0x7f71d81a36a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:26.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.056+0000 7f71df6fa640 1 --2- 172.21.15.83:0/2788313351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71d81018e0 0x7f71d819f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:26.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.056+0000 7f71df6fa640 1 --2- 172.21.15.83:0/2788313351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71d8100fc0 0x7f71d819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:26.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.056+0000 7f71df6fa640 1 -- 172.21.15.83:0/2788313351 >> 172.21.15.83:0/2788313351 conn(0x7f71d8077f50 msgr2=0x7f71d8103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:26.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.056+0000 7f71df6fa640 1 -- 172.21.15.83:0/2788313351 shutdown_connections 2024-04-25T11:54:26.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.056+0000 7f71df6fa640 1 -- 172.21.15.83:0/2788313351 wait complete. 2024-04-25T11:54:26.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-25T11:54:26.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T11:54:26.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T11:54:26.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: Cluster is now healthy 2024-04-25T11:54:26.058 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/3934589586' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T11:54:26.059 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.83:6838/325778165,v1:172.21.15.83:6841/325778165] up:boot 2024-04-25T11:54:26.059 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.83:6839/1778381896,v1:172.21.15.83:6840/1778381896] up:boot 2024-04-25T11:54:26.059 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.94:6833/1576549222,v1:172.21.15.94:6835/1576549222] up:boot 2024-04-25T11:54:26.059 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.94:6832/325114104,v1:172.21.15.94:6834/325114104] up:boot 2024-04-25T11:54:26.059 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.94:6836/3617346119,v1:172.21.15.94:6837/3617346119] up:boot 2024-04-25T11:54:26.059 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.94:6838/3681861560,v1:172.21.15.94:6839/3681861560] up:boot 2024-04-25T11:54:26.060 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.111:6832/703193743,v1:172.21.15.111:6834/703193743] up:boot 2024-04-25T11:54:26.060 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.111:6833/2681315840,v1:172.21.15.111:6835/2681315840] up:boot 2024-04-25T11:54:26.060 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.111:6836/910979687,v1:172.21.15.111:6838/910979687] up:boot 2024-04-25T11:54:26.060 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: mds.? [v2:172.21.15.111:6837/4041192626,v1:172.21.15.111:6839/4041192626] up:boot 2024-04-25T11:54:26.060 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: fsmap 12 up:standby 2024-04-25T11:54:26.061 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T11:54:26.061 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T11:54:26.061 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T11:54:26.061 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:54:26.061 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T11:54:26.061 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:54:26.062 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T11:54:26.062 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T11:54:26.062 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T11:54:26.062 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T11:54:26.062 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:25 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2788313351' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:54:26.063 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:54:26.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: pgmap v336: 97 pgs: 97 active+clean; 1.3 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-25T11:54:26.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T11:54:26.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T11:54:26.074 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: Cluster is now healthy 2024-04-25T11:54:26.075 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/3934589586' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T11:54:26.075 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.83:6838/325778165,v1:172.21.15.83:6841/325778165] up:boot 2024-04-25T11:54:26.075 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.83:6839/1778381896,v1:172.21.15.83:6840/1778381896] up:boot 2024-04-25T11:54:26.075 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.94:6833/1576549222,v1:172.21.15.94:6835/1576549222] up:boot 2024-04-25T11:54:26.075 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.94:6832/325114104,v1:172.21.15.94:6834/325114104] up:boot 2024-04-25T11:54:26.075 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.94:6836/3617346119,v1:172.21.15.94:6837/3617346119] up:boot 2024-04-25T11:54:26.075 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.94:6838/3681861560,v1:172.21.15.94:6839/3681861560] up:boot 2024-04-25T11:54:26.075 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.111:6832/703193743,v1:172.21.15.111:6834/703193743] up:boot 2024-04-25T11:54:26.076 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.111:6833/2681315840,v1:172.21.15.111:6835/2681315840] up:boot 2024-04-25T11:54:26.076 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.111:6836/910979687,v1:172.21.15.111:6838/910979687] up:boot 2024-04-25T11:54:26.076 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: mds.? [v2:172.21.15.111:6837/4041192626,v1:172.21.15.111:6839/4041192626] up:boot 2024-04-25T11:54:26.076 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: fsmap 12 up:standby 2024-04-25T11:54:26.076 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T11:54:26.076 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T11:54:26.076 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T11:54:26.076 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T11:54:26.076 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T11:54:26.077 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T11:54:26.077 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T11:54:26.077 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T11:54:26.077 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T11:54:26.077 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: from='mgr.15360 172.21.15.83:0/3372978950' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T11:54:26.077 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:25 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2788313351' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:54:26.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.183+0000 7f66c2d51640 1 -- 172.21.15.83:0/1020931838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66bc1010a0 msgr2=0x7f66bc101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:26.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.183+0000 7f66c2d51640 1 --2- 172.21.15.83:0/1020931838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66bc1010a0 0x7f66bc101480 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009900 tx=0x7f66b002a410 comp rx=0 tx=0).stop 2024-04-25T11:54:26.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.183+0000 7f66c2d51640 1 -- 172.21.15.83:0/1020931838 shutdown_connections 2024-04-25T11:54:26.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.183+0000 7f66c2d51640 1 --2- 172.21.15.83:0/1020931838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66bc106170 0x7f66bc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:26.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.183+0000 7f66c2d51640 1 --2- 172.21.15.83:0/1020931838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66bc101a50 0x7f66bc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:26.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.183+0000 7f66c2d51640 1 --2- 172.21.15.83:0/1020931838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66bc1010a0 0x7f66bc101480 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:26.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.183+0000 7f66c2d51640 1 -- 172.21.15.83:0/1020931838 >> 172.21.15.83:0/1020931838 conn(0x7f66bc077eb0 msgr2=0x7f66bc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:26.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.183+0000 7f66c2d51640 1 -- 172.21.15.83:0/1020931838 shutdown_connections 2024-04-25T11:54:26.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.183+0000 7f66c2d51640 1 -- 172.21.15.83:0/1020931838 wait complete. 2024-04-25T11:54:26.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.183+0000 7f66c2d51640 1 Processor -- start 2024-04-25T11:54:26.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.183+0000 7f66c2d51640 1 -- start start 2024-04-25T11:54:26.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c2d51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66bc1010a0 0x7f66bc198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:26.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c2d51640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66bc106170 0x7f66bc198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:26.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c2d51640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66bc19cf50 0x7f66bc19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c2d51640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc1121f0 con 0x7f66bc1010a0 2024-04-25T11:54:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c2d51640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc112070 con 0x7f66bc106170 2024-04-25T11:54:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c2d51640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc112370 con 0x7f66bc19cf50 2024-04-25T11:54:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c0aed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66bc1010a0 0x7f66bc198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c0aed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66bc1010a0 0x7f66bc198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38048/0 (socket says 172.21.15.83:38048) 2024-04-25T11:54:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c0aed640 1 -- 172.21.15.83:0/1220389792 learned_addr learned my addr 172.21.15.83:0/1220389792 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-25T11:54:26.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66bbfff640 1 --2- 172.21.15.83:0/1220389792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66bc106170 0x7f66bc198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c12ee640 1 --2- 172.21.15.83:0/1220389792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66bc19cf50 0x7f66bc19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c0aed640 1 -- 172.21.15.83:0/1220389792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66bc19cf50 msgr2=0x7f66bc19d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c0aed640 1 --2- 172.21.15.83:0/1220389792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66bc19cf50 0x7f66bc19d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c0aed640 1 -- 172.21.15.83:0/1220389792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66bc106170 msgr2=0x7f66bc198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c0aed640 1 --2- 172.21.15.83:0/1220389792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66bc106170 0x7f66bc198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c0aed640 1 -- 172.21.15.83:0/1220389792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc19da40 con 0x7f66bc1010a0 2024-04-25T11:54:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c0aed640 1 --2- 172.21.15.83:0/1220389792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66bc1010a0 0x7f66bc198690 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f66b0005940 tx=0x7f66b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66b9ffb640 1 -- 172.21.15.83:0/1220389792 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b003f030 con 0x7f66bc1010a0 2024-04-25T11:54:26.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66b9ffb640 1 -- 172.21.15.83:0/1220389792 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b0003cb0 con 0x7f66bc1010a0 2024-04-25T11:54:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66c2d51640 1 -- 172.21.15.83:0/1220389792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66bc19dcd0 con 0x7f66bc1010a0 2024-04-25T11:54:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.184+0000 7f66b9ffb640 1 -- 172.21.15.83:0/1220389792 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b0035070 con 0x7f66bc1010a0 2024-04-25T11:54:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.185+0000 7f66c2d51640 1 -- 172.21.15.83:0/1220389792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66bc1a5490 con 0x7f66bc1010a0 2024-04-25T11:54:26.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.185+0000 7f66c2d51640 1 -- 172.21.15.83:0/1220389792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66bc0691c0 con 0x7f66bc1010a0 2024-04-25T11:54:26.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.186+0000 7f66b9ffb640 1 -- 172.21.15.83:0/1220389792 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f66b0042030 con 0x7f66bc1010a0 2024-04-25T11:54:26.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.187+0000 7f66b9ffb640 1 --2- 172.21.15.83:0/1220389792 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f66ac0a3d20 0x7f66ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:54:26.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.187+0000 7f66bbfff640 1 --2- 172.21.15.83:0/1220389792 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f66ac0a3d20 0x7f66ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:54:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.187+0000 7f66b9ffb640 1 -- 172.21.15.83:0/1220389792 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8403+0+0 (secure 0 0 0) 0x7f66b00f0f70 con 0x7f66bc1010a0 2024-04-25T11:54:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.188+0000 7f66bbfff640 1 --2- 172.21.15.83:0/1220389792 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f66ac0a3d20 0x7f66ac0a61e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66bc105bf0 tx=0x7f66a800a2d0 comp rx=0 tx=0).ready entity=mgr.15360 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:54:26.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.192+0000 7f66b9ffb640 1 -- 172.21.15.83:0/1220389792 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66b00bd680 con 0x7f66bc1010a0 2024-04-25T11:54:26.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:26.371+0000 7f66c2d51640 1 -- 172.21.15.83:0/1220389792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f66bc101a50 con 0x7f66bc1010a0 2024-04-25T11:54:27.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:27.058+0000 7f66b9ffb640 1 -- 172.21.15.83:0/1220389792 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) ==== 169+0+0 (secure 0 0 0) 0x7f66b00bcf80 con 0x7f66bc1010a0 2024-04-25T11:54:27.048 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-25T11:54:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:27.062+0000 7f66c2d51640 1 -- 172.21.15.83:0/1220389792 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f66ac0a3d20 msgr2=0x7f66ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:27.062+0000 7f66c2d51640 1 --2- 172.21.15.83:0/1220389792 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f66ac0a3d20 0x7f66ac0a61e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66bc105bf0 tx=0x7f66a800a2d0 comp rx=0 tx=0).stop 2024-04-25T11:54:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:27.062+0000 7f66c2d51640 1 -- 172.21.15.83:0/1220389792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66bc1010a0 msgr2=0x7f66bc198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:54:27.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:27.062+0000 7f66c2d51640 1 --2- 172.21.15.83:0/1220389792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66bc1010a0 0x7f66bc198690 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f66b0005940 tx=0x7f66b00042f0 comp rx=0 tx=0).stop 2024-04-25T11:54:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:27.062+0000 7f66c2d51640 1 -- 172.21.15.83:0/1220389792 shutdown_connections 2024-04-25T11:54:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:27.062+0000 7f66c2d51640 1 --2- 172.21.15.83:0/1220389792 >> [v2:172.21.15.83:6800/4112952368,v1:172.21.15.83:6801/4112952368] conn(0x7f66ac0a3d20 0x7f66ac0a61e0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66bc105bf0 tx=0x7f66a800a2d0 comp rx=0 tx=0).stop 2024-04-25T11:54:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:27.062+0000 7f66c2d51640 1 --2- 172.21.15.83:0/1220389792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66bc19cf50 0x7f66bc19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:27.062+0000 7f66c2d51640 1 --2- 172.21.15.83:0/1220389792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66bc106170 0x7f66bc198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:54:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:27.062+0000 7f66c2d51640 1 --2- 172.21.15.83:0/1220389792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66bc1010a0 0x7f66bc198690 secure :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f66b0005940 tx=0x7f66b00042f0 comp rx=0 tx=0).stop 2024-04-25T11:54:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:27.062+0000 7f66c2d51640 1 -- 172.21.15.83:0/1220389792 >> 172.21.15.83:0/1220389792 conn(0x7f66bc077eb0 msgr2=0x7f66bc100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:54:27.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:27.063+0000 7f66c2d51640 1 -- 172.21.15.83:0/1220389792 shutdown_connections 2024-04-25T11:54:27.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-25T11:54:27.063+0000 7f66c2d51640 1 -- 172.21.15.83:0/1220389792 wait complete. 2024-04-25T11:54:27.067 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:54:27.094 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:54:27.127 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:54:27.159 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T11:54:27.160 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T11:54:27.179 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T11:54:27.197 DEBUG:teuthology.orchestra.run.smithi111:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T11:54:27.217 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T11:54:27.217 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T11:54:27.217 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.a 2024-04-25T11:54:27.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:27 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/2788313351' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T11:54:27.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:27 smithi083 ceph-mon[29330]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T11:54:27.309 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:27 smithi083 ceph-mon[29330]: from='client.? 172.21.15.83:0/1220389792' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:54:27.310 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:27 smithi083 systemd[1]: Stopping Ceph mon.a for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:54:27.323 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:27 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/2788313351' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T11:54:27.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:27 smithi111 ceph-mon[32784]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T11:54:27.324 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:27 smithi111 ceph-mon[32784]: from='client.? 172.21.15.83:0/1220389792' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:54:27.524 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:27 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/2788313351' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T11:54:27.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:27 smithi094 ceph-mon[33032]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T11:54:27.525 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:27 smithi094 ceph-mon[33032]: from='client.? 172.21.15.83:0/1220389792' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T11:54:27.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a[29326]: 2024-04-25T11:54:27.540+0000 7f332555b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:54:27.559 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a[29326]: 2024-04-25T11:54:27.540+0000 7f332555b700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T11:54:27.863 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:27 smithi083 podman[87716]: 2024-04-25 11:54:27.571211564 +0000 UTC m=+0.152122075 container died b35e79aacc4caa453c37b1077e06f233a5a06ac3bdda6cd43828836179f78708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-25T11:54:28.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:28 smithi083 podman[87716]: 2024-04-25 11:54:28.024384216 +0000 UTC m=+0.605294727 container cleanup b35e79aacc4caa453c37b1077e06f233a5a06ac3bdda6cd43828836179f78708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-25T11:54:28.157 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:28 smithi083 bash[87716]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a 2024-04-25T11:54:28.461 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T11:54:28.483 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:28 smithi083 podman[87731]: 2024-04-25 11:54:28.309453735 +0000 UTC m=+0.740434704 container remove b35e79aacc4caa453c37b1077e06f233a5a06ac3bdda6cd43828836179f78708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:54:28.483 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:28 smithi083 systemd[1]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T11:54:28.484 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:28 smithi083 systemd[1]: Stopped Ceph mon.a for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:54:28.484 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 25 11:54:28 smithi083 systemd[1]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.a.service: Consumed 17.442s CPU time. 2024-04-25T11:54:28.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:54:28.509 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T11:54:28.509 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T11:54:28.509 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.b 2024-04-25T11:54:28.810 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:28 smithi094 systemd[1]: Stopping Ceph mon.b for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:54:29.274 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:28 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-b[33028]: 2024-04-25T11:54:28.821+0000 7f50e627c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:54:29.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:28 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-b[33028]: 2024-04-25T11:54:28.821+0000 7f50e627c700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T11:54:29.275 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:29 smithi094 podman[68699]: 2024-04-25 11:54:29.025708374 +0000 UTC m=+0.312270457 container died 5643b901062139cde30ea9065870f8fdac0a946c79f3e1f4bb073130b52b2a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:54:29.652 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:29 smithi094 podman[68699]: 2024-04-25 11:54:29.379463064 +0000 UTC m=+0.666025156 container cleanup 5643b901062139cde30ea9065870f8fdac0a946c79f3e1f4bb073130b52b2a49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-25T11:54:29.652 INFO:journalctl@ceph.mon.b.smithi094.stdout:Apr 25 11:54:29 smithi094 bash[68699]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-b 2024-04-25T11:54:29.798 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T11:54:29.848 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:54:29.848 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T11:54:29.848 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T11:54:29.849 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.c 2024-04-25T11:54:30.146 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:29 smithi111 systemd[1]: Stopping Ceph mon.c for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:54:30.480 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:30 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-c[32780]: 2024-04-25T11:54:30.156+0000 7fa2e9b4f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:54:30.480 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:30 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-c[32780]: 2024-04-25T11:54:30.156+0000 7fa2e9b4f700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T11:54:30.480 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:30 smithi111 podman[69178]: 2024-04-25 11:54:30.228770943 +0000 UTC m=+0.177192612 container died a736b2f2b4a60f2333a181624ff0f2589401f36c8ca19d4c24e55a206e20f355 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:54:30.823 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:30 smithi111 podman[69178]: 2024-04-25 11:54:30.590697275 +0000 UTC m=+0.539118950 container cleanup a736b2f2b4a60f2333a181624ff0f2589401f36c8ca19d4c24e55a206e20f355 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T11:54:30.824 INFO:journalctl@ceph.mon.c.smithi111.stdout:Apr 25 11:54:30 smithi111 bash[69178]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mon-c 2024-04-25T11:54:31.005 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T11:54:31.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:54:31.057 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T11:54:31.057 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T11:54:31.057 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.x 2024-04-25T11:54:31.385 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:54:31 smithi083 systemd[1]: Stopping Ceph mgr.x for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:54:31.658 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:54:31 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x[29555]: 2024-04-25T11:54:31.395+0000 7efeb5ba3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:54:31.658 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:54:31 smithi083 podman[87848]: 2024-04-25 11:54:31.426759327 +0000 UTC m=+0.159345919 container died 6c82361a54d3791a38c5ae10b7af661a7050784d8a228600cab53808a563ab2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T11:54:32.044 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:54:31 smithi083 podman[87848]: 2024-04-25 11:54:31.830034102 +0000 UTC m=+0.562620740 container cleanup 6c82361a54d3791a38c5ae10b7af661a7050784d8a228600cab53808a563ab2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T11:54:32.045 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 25 11:54:31 smithi083 bash[87848]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-x 2024-04-25T11:54:32.154 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T11:54:32.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:54:32.189 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T11:54:32.190 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T11:54:32.190 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.y 2024-04-25T11:54:32.500 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:54:32 smithi094 systemd[1]: Stopping Ceph mgr.y for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:54:32.774 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:54:32 smithi094 podman[68823]: 2024-04-25 11:54:32.512369529 +0000 UTC m=+0.124371086 container died 6b433dab8161c6375b436e7e3407f21d7c84384a14424bdaf9c32952dbdb7752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T11:54:33.223 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:54:33 smithi094 podman[68823]: 2024-04-25 11:54:33.025523498 +0000 UTC m=+0.637525038 container cleanup 6b433dab8161c6375b436e7e3407f21d7c84384a14424bdaf9c32952dbdb7752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:54:33.223 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:54:33 smithi094 bash[68823]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y 2024-04-25T11:54:33.457 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T11:54:33.477 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:54:33 smithi094 podman[68836]: 2024-04-25 11:54:33.235216543 +0000 UTC m=+0.718724050 container remove 6b433dab8161c6375b436e7e3407f21d7c84384a14424bdaf9c32952dbdb7752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T11:54:33.478 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:54:33 smithi094 systemd[1]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-25T11:54:33.478 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:54:33 smithi094 systemd[1]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-04-25T11:54:33.478 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:54:33 smithi094 systemd[1]: Stopped Ceph mgr.y for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:54:33.478 INFO:journalctl@ceph.mgr.y.smithi094.stdout:Apr 25 11:54:33 smithi094 systemd[1]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.y.service: Consumed 20.078s CPU time. 2024-04-25T11:54:33.490 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:54:33.490 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T11:54:33.491 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T11:54:33.491 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.z 2024-04-25T11:54:33.823 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:54:33 smithi111 systemd[1]: Stopping Ceph mgr.z for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:54:34.101 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:54:33 smithi111 podman[69299]: 2024-04-25 11:54:33.840523707 +0000 UTC m=+0.149894271 container died 3983775f5066404d461acf8727ef80295d6686cc64a278fdf6e242153dd2f77c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T11:54:34.492 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:54:34 smithi111 podman[69299]: 2024-04-25 11:54:34.220024271 +0000 UTC m=+0.529394843 container cleanup 3983775f5066404d461acf8727ef80295d6686cc64a278fdf6e242153dd2f77c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T11:54:34.493 INFO:journalctl@ceph.mgr.z.smithi111.stdout:Apr 25 11:54:34 smithi111 bash[69299]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mgr-z 2024-04-25T11:54:34.623 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T11:54:34.657 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:54:34.657 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T11:54:34.657 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T11:54:34.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.0 2024-04-25T11:54:34.956 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:54:34 smithi083 systemd[1]: Stopping Ceph osd.0 for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:54:35.407 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:54:34 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-0[44082]: 2024-04-25T11:54:34.967+0000 7fe4dde50700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:54:35.408 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:54:34 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-0[44082]: 2024-04-25T11:54:34.967+0000 7fe4dde50700 -1 osd.0 102 *** Got signal Terminated *** 2024-04-25T11:54:35.408 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:54:34 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-0[44082]: 2024-04-25T11:54:34.967+0000 7fe4dde50700 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:54:40.643 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:54:40 smithi083 podman[87970]: 2024-04-25 11:54:40.322269127 +0000 UTC m=+5.461004740 container died 027c79d7f018322c27aae51a1a9bc842890014f26f80b4396e5c0077bd98b292 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T11:54:40.907 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:54:40 smithi083 podman[87970]: 2024-04-25 11:54:40.65082293 +0000 UTC m=+5.789558525 container cleanup 027c79d7f018322c27aae51a1a9bc842890014f26f80b4396e5c0077bd98b292 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:54:40.908 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:54:40 smithi083 bash[87970]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-0 2024-04-25T11:54:40.908 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:54:40 smithi083 podman[87987]: 2024-04-25 11:54:40.869167738 +0000 UTC m=+0.549032415 container remove 027c79d7f018322c27aae51a1a9bc842890014f26f80b4396e5c0077bd98b292 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T11:54:41.569 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:54:41 smithi083 podman[88082]: 2024-04-25 11:54:41.218030306 +0000 UTC m=+0.017869506 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:54:41.569 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:54:41 smithi083 podman[88082]: 2024-04-25 11:54:41.328369539 +0000 UTC m=+0.128208728 container create 4cddf6923ab46c9c3ea61cfba396555d26770726b030be09e9dc77f5819ba78a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T11:54:41.569 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:54:41 smithi083 podman[88082]: 2024-04-25 11:54:41.581035 +0000 UTC m=+0.380874186 container init 4cddf6923ab46c9c3ea61cfba396555d26770726b030be09e9dc77f5819ba78a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T11:54:41.866 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:54:41 smithi083 podman[88082]: 2024-04-25 11:54:41.584484852 +0000 UTC m=+0.384324039 container start 4cddf6923ab46c9c3ea61cfba396555d26770726b030be09e9dc77f5819ba78a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T11:54:41.866 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:54:41 smithi083 podman[88082]: 2024-04-25 11:54:41.672393325 +0000 UTC m=+0.472232539 container attach 4cddf6923ab46c9c3ea61cfba396555d26770726b030be09e9dc77f5819ba78a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T11:54:42.155 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 25 11:54:41 smithi083 podman[88082]: 2024-04-25 11:54:41.877136026 +0000 UTC m=+0.676975209 container died 4cddf6923ab46c9c3ea61cfba396555d26770726b030be09e9dc77f5819ba78a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T11:54:42.479 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T11:54:42.519 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:54:42.520 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T11:54:42.520 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T11:54:42.520 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.1 2024-04-25T11:54:42.836 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:54:42 smithi094 systemd[1]: Stopping Ceph osd.1 for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:54:43.113 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:54:42 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-1[37640]: 2024-04-25T11:54:42.846+0000 7faa2adb7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:54:43.113 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:54:42 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-1[37640]: 2024-04-25T11:54:42.846+0000 7faa2adb7700 -1 osd.1 102 *** Got signal Terminated *** 2024-04-25T11:54:43.113 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:54:42 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-1[37640]: 2024-04-25T11:54:42.846+0000 7faa2adb7700 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:54:48.519 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:54:48 smithi094 podman[68958]: 2024-04-25 11:54:48.247064432 +0000 UTC m=+5.533463932 container died 4f664b7fa4f544e4951c166ba9faaad63d756e8ef0522a365d91a0bf33062109 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:54:48.775 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:54:48 smithi094 podman[68958]: 2024-04-25 11:54:48.629769479 +0000 UTC m=+5.916168987 container cleanup 4f664b7fa4f544e4951c166ba9faaad63d756e8ef0522a365d91a0bf33062109 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T11:54:48.775 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:54:48 smithi094 bash[68958]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-1 2024-04-25T11:54:49.244 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:54:48 smithi094 podman[68976]: 2024-04-25 11:54:48.914249587 +0000 UTC m=+0.668679974 container remove 4f664b7fa4f544e4951c166ba9faaad63d756e8ef0522a365d91a0bf33062109 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T11:54:49.507 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:54:49 smithi094 podman[69068]: 2024-04-25 11:54:49.158732241 +0000 UTC m=+0.017867326 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:54:49.507 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:54:49 smithi094 podman[69068]: 2024-04-25 11:54:49.256605689 +0000 UTC m=+0.115740780 container create 69afd8b05a98aee19be104bbfc70706a8fdd08bc9ebc400b7ab08c6c73d17a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:54:49.775 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:54:49 smithi094 podman[69068]: 2024-04-25 11:54:49.518607051 +0000 UTC m=+0.377742136 container init 69afd8b05a98aee19be104bbfc70706a8fdd08bc9ebc400b7ab08c6c73d17a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T11:54:49.776 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:54:49 smithi094 podman[69068]: 2024-04-25 11:54:49.52183193 +0000 UTC m=+0.380967024 container start 69afd8b05a98aee19be104bbfc70706a8fdd08bc9ebc400b7ab08c6c73d17a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T11:54:49.776 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:54:49 smithi094 podman[69068]: 2024-04-25 11:54:49.601768077 +0000 UTC m=+0.460903163 container attach 69afd8b05a98aee19be104bbfc70706a8fdd08bc9ebc400b7ab08c6c73d17a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T11:54:50.110 INFO:journalctl@ceph.osd.1.smithi094.stdout:Apr 25 11:54:49 smithi094 podman[69068]: 2024-04-25 11:54:49.816200808 +0000 UTC m=+0.675335894 container died 69afd8b05a98aee19be104bbfc70706a8fdd08bc9ebc400b7ab08c6c73d17a0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:54:50.437 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T11:54:50.478 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:54:50.478 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T11:54:50.479 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T11:54:50.479 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.2 2024-04-25T11:54:50.788 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:54:50 smithi111 systemd[1]: Stopping Ceph osd.2 for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:54:51.074 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:54:50 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-2[37914]: 2024-04-25T11:54:50.798+0000 7ff7f84be700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:54:51.074 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:54:50 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-2[37914]: 2024-04-25T11:54:50.798+0000 7ff7f84be700 -1 osd.2 102 *** Got signal Terminated *** 2024-04-25T11:54:51.075 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:54:50 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-2[37914]: 2024-04-25T11:54:50.798+0000 7ff7f84be700 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:54:56.564 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:54:56 smithi111 podman[69435]: 2024-04-25 11:54:56.294297935 +0000 UTC m=+5.614745958 container died 73cc4e57e47b6180fc2804b3fd0867aaef2a8f6b613758d3de247ccbc389b4f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T11:54:56.824 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:54:56 smithi111 podman[69435]: 2024-04-25 11:54:56.656830709 +0000 UTC m=+5.977278726 container cleanup 73cc4e57e47b6180fc2804b3fd0867aaef2a8f6b613758d3de247ccbc389b4f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T11:54:56.824 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:54:56 smithi111 bash[69435]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-2 2024-04-25T11:54:57.301 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:54:56 smithi111 podman[69453]: 2024-04-25 11:54:56.966837671 +0000 UTC m=+0.666272448 container remove 73cc4e57e47b6180fc2804b3fd0867aaef2a8f6b613758d3de247ccbc389b4f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T11:54:57.567 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:54:57 smithi111 podman[69543]: 2024-04-25 11:54:57.215097674 +0000 UTC m=+0.017692658 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:54:57.567 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:54:57 smithi111 podman[69543]: 2024-04-25 11:54:57.342630776 +0000 UTC m=+0.145225773 container create 498f710236b618ed596a42dbd09498babf7aa2d89a8a53f9a634b8bbc48731d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:54:57.567 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:54:57 smithi111 podman[69543]: 2024-04-25 11:54:57.578253767 +0000 UTC m=+0.380848723 container init 498f710236b618ed596a42dbd09498babf7aa2d89a8a53f9a634b8bbc48731d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:54:57.824 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:54:57 smithi111 podman[69543]: 2024-04-25 11:54:57.58144379 +0000 UTC m=+0.384038747 container start 498f710236b618ed596a42dbd09498babf7aa2d89a8a53f9a634b8bbc48731d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:54:57.824 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:54:57 smithi111 podman[69543]: 2024-04-25 11:54:57.686799244 +0000 UTC m=+0.489394213 container attach 498f710236b618ed596a42dbd09498babf7aa2d89a8a53f9a634b8bbc48731d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T11:54:58.187 INFO:journalctl@ceph.osd.2.smithi111.stdout:Apr 25 11:54:57 smithi111 podman[69543]: 2024-04-25 11:54:57.887329749 +0000 UTC m=+0.689924716 container died 498f710236b618ed596a42dbd09498babf7aa2d89a8a53f9a634b8bbc48731d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T11:54:58.544 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T11:54:58.582 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:54:58.583 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T11:54:58.583 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T11:54:58.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.3 2024-04-25T11:54:58.892 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:54:58 smithi083 systemd[1]: Stopping Ceph osd.3 for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:54:59.154 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:54:58 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-3[51025]: 2024-04-25T11:54:58.903+0000 7fbb79047700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:54:59.154 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:54:58 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-3[51025]: 2024-04-25T11:54:58.903+0000 7fbb79047700 -1 osd.3 102 *** Got signal Terminated *** 2024-04-25T11:54:59.154 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:54:58 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-3[51025]: 2024-04-25T11:54:58.903+0000 7fbb79047700 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:55:01.408 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:00 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:00.965+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:02.408 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:01 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:01.988+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:03.408 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:02 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:02.986+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:03.574 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:03 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:03.327+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:03.775 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:03 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:03.468+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:04.323 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:04 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:04.009+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:04.324 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:03 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:03.946+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:04.642 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:04 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:04.441+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:04.643 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:55:04 smithi083 podman[88189]: 2024-04-25 11:55:04.333526743 +0000 UTC m=+5.549962405 container died 29afb0f84455166254809c712f1c50463ba37b2153f58e68719708dadadd6214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T11:55:04.643 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:55:04 smithi083 podman[88189]: 2024-04-25 11:55:04.653367356 +0000 UTC m=+5.869803020 container cleanup 29afb0f84455166254809c712f1c50463ba37b2153f58e68719708dadadd6214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T11:55:04.775 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:04 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:04.513+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:04.824 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:04 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:04.369+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:04.908 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:55:04 smithi083 bash[88189]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-3 2024-04-25T11:55:05.324 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:04 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:04.897+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:05.408 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:55:04 smithi083 podman[88218]: 2024-04-25 11:55:04.946413834 +0000 UTC m=+0.614161380 container remove 29afb0f84455166254809c712f1c50463ba37b2153f58e68719708dadadd6214 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T11:55:05.408 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:05 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:05.012+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:05.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:05 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:05.420+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:05.658 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:55:05 smithi083 podman[88307]: 2024-04-25 11:55:05.32208797 +0000 UTC m=+0.018514401 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:55:05.659 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:55:05 smithi083 podman[88307]: 2024-04-25 11:55:05.430516175 +0000 UTC m=+0.126942577 container create d22e804b04056c388bf6b774ed6c06ae1e2350a1812c1fad7041c3431e7e4a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T11:55:05.659 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:55:05 smithi083 podman[88307]: 2024-04-25 11:55:05.666232591 +0000 UTC m=+0.362658995 container init d22e804b04056c388bf6b774ed6c06ae1e2350a1812c1fad7041c3431e7e4a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T11:55:05.659 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:55:05 smithi083 podman[88307]: 2024-04-25 11:55:05.669324507 +0000 UTC m=+0.365750923 container start d22e804b04056c388bf6b774ed6c06ae1e2350a1812c1fad7041c3431e7e4a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T11:55:05.775 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:05 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:05.516+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:05.825 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:05 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:05.353+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:05.908 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:55:05 smithi083 podman[88307]: 2024-04-25 11:55:05.758453898 +0000 UTC m=+0.454880315 container attach d22e804b04056c388bf6b774ed6c06ae1e2350a1812c1fad7041c3431e7e4a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T11:55:06.247 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:06 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:06.023+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:06.247 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 25 11:55:05 smithi083 podman[88307]: 2024-04-25 11:55:05.963652845 +0000 UTC m=+0.660079264 container died d22e804b04056c388bf6b774ed6c06ae1e2350a1812c1fad7041c3431e7e4a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T11:55:06.327 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:05 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:05.920+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:06.509 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:06 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:06.386+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:06.538 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T11:55:06.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:55:06.576 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T11:55:06.577 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T11:55:06.577 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.4 2024-04-25T11:55:06.775 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:06 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:06.499+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:06.775 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:55:06 smithi094 systemd[1]: Stopping Ceph osd.4 for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:55:06.824 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:06 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:06.397+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:07.275 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:55:06 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-4[44148]: 2024-04-25T11:55:06.916+0000 7f39a3e13700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:55:07.276 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:55:06 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-4[44148]: 2024-04-25T11:55:06.916+0000 7f39a3e13700 -1 osd.4 102 *** Got signal Terminated *** 2024-04-25T11:55:07.276 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:55:06 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-4[44148]: 2024-04-25T11:55:06.916+0000 7f39a3e13700 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:55:07.324 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:06 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:06.938+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:07.408 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:07 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:07.030+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:07.775 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:07 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:07.527+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:07.824 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:07 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:07.404+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:07.908 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:07 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:07.435+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:08.324 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:07 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:07.888+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:08.408 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:08 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:08.016+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:08.743 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:08 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:08.383+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:08.775 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:08 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:08.508+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:08.908 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:08 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:08.453+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:09.074 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:08 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:08.752+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:09.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:08 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:08.848+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:09.395 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:08 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:08.979+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:09.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:09 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:09.406+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:09.741 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:09 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:09.432+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:09.741 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:09 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:09.432+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:44.108661+0000 front 2024-04-25T11:54:44.108761+0000 (oldest deadline 2024-04-25T11:55:09.408699+0000) 2024-04-25T11:55:09.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:09 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:09.462+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:09.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:09 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:09.462+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:10.074 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:09 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:09.752+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:10.074 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:09 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:09.859+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:10.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:09 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:09.859+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:10.375 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:09 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:09.964+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:10.375 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:09 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:09.964+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:10.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:10 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:10.385+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:10.708 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:10 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:10.407+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:10.708 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:10 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:10.407+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:44.108661+0000 front 2024-04-25T11:54:44.108761+0000 (oldest deadline 2024-04-25T11:55:09.408699+0000) 2024-04-25T11:55:10.775 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:10 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:10.500+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:10.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:10 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:10.500+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:11.074 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:10 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:10.719+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:11.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:10 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:10.851+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:11.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:10 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:10.851+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:11.342 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:10 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:10.964+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:11.342 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:10 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:10.964+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:11.467 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:55:11 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-4[44148]: 2024-04-25T11:55:11.118+0000 7f39a0407700 -1 osd.4 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:44.851140+0000 front 2024-04-25T11:54:44.851107+0000 (oldest deadline 2024-04-25T11:55:10.751310+0000) 2024-04-25T11:55:11.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:11 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:11.352+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:11.709 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:11 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:11.386+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:11.709 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:11 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:11.386+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:44.108661+0000 front 2024-04-25T11:54:44.108761+0000 (oldest deadline 2024-04-25T11:55:09.408699+0000) 2024-04-25T11:55:11.775 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:11 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:11.477+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:11.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:11 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:11.477+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:12.074 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:11 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:11.720+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:12.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:11 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:11.870+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:12.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:11 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:11.870+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:12.312 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:12 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:12.005+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:12.313 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:12 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:12.005+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:12.525 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:55:12 smithi094 podman[69174]: 2024-04-25 11:55:12.281724955 +0000 UTC m=+5.486235195 container died f1a32c4b5cc075653ad2baf64e7a60d388595f33e6ec08c244fa98a89019febf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T11:55:12.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:12 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:12.512+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:12.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:12 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:12.512+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:12.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:12 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:12.323+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:12.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:12 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:12.323+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:12.756 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:12 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:12.374+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:12.756 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:12 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:12.374+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:44.108661+0000 front 2024-04-25T11:54:44.108761+0000 (oldest deadline 2024-04-25T11:55:09.408699+0000) 2024-04-25T11:55:12.932 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:55:12 smithi094 podman[69174]: 2024-04-25 11:55:12.633952941 +0000 UTC m=+5.838463178 container cleanup f1a32c4b5cc075653ad2baf64e7a60d388595f33e6ec08c244fa98a89019febf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T11:55:12.932 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:55:12 smithi094 bash[69174]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-4 2024-04-25T11:55:13.074 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:12 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:12.766+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:13.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:12 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:12.848+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:13.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:12 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:12.848+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:13.275 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:55:12 smithi094 podman[69190]: 2024-04-25 11:55:12.943197684 +0000 UTC m=+0.661315656 container remove f1a32c4b5cc075653ad2baf64e7a60d388595f33e6ec08c244fa98a89019febf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T11:55:13.296 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:12 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:12.992+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:13.296 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:12 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:12.992+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:13.537 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:13 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:13.291+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:13.537 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:55:13 smithi094 podman[69285]: 2024-04-25 11:55:13.207486372 +0000 UTC m=+0.017812206 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:55:13.538 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:55:13 smithi094 podman[69285]: 2024-04-25 11:55:13.318987637 +0000 UTC m=+0.129313455 container create be40fe990343e8cb4f8deba9312c75f1de696c1078cae670ef63f81d2ba6a5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T11:55:13.538 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:55:13 smithi094 podman[69285]: 2024-04-25 11:55:13.522105452 +0000 UTC m=+0.332431270 container init be40fe990343e8cb4f8deba9312c75f1de696c1078cae670ef63f81d2ba6a5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-25T11:55:13.538 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:55:13 smithi094 podman[69285]: 2024-04-25 11:55:13.525066878 +0000 UTC m=+0.335392701 container start be40fe990343e8cb4f8deba9312c75f1de696c1078cae670ef63f81d2ba6a5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T11:55:13.574 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:13 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:13.374+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:13.575 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:13 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:13.374+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:44.108661+0000 front 2024-04-25T11:54:44.108761+0000 (oldest deadline 2024-04-25T11:55:09.408699+0000) 2024-04-25T11:55:13.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:13 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:13.306+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:13.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:13 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:13.306+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:13.808 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:55:13 smithi094 podman[69285]: 2024-04-25 11:55:13.605537652 +0000 UTC m=+0.415863478 container attach be40fe990343e8cb4f8deba9312c75f1de696c1078cae670ef63f81d2ba6a5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-25T11:55:13.808 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:13 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:13.546+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:13.808 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:13 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:13.546+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:14.074 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:13 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:13.796+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:14.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:13 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:13.866+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:14.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:13 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:13.866+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:14.105 INFO:journalctl@ceph.osd.4.smithi094.stdout:Apr 25 11:55:13 smithi094 podman[69285]: 2024-04-25 11:55:13.819294691 +0000 UTC m=+0.629620520 container died be40fe990343e8cb4f8deba9312c75f1de696c1078cae670ef63f81d2ba6a5e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T11:55:14.271 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:14 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:14.007+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:14.272 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:14 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:14.007+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:14.383 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:14 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:14.319+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:14.403 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T11:55:14.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:55:14.482 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T11:55:14.482 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T11:55:14.482 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.5 2024-04-25T11:55:14.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:14 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:14.281+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:14.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:14 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:14.281+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:14.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:14 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:14.557+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:14.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:14 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:14.557+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:14.782 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:14 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:14.347+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:14.782 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:14 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:14.347+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:44.108661+0000 front 2024-04-25T11:54:44.108761+0000 (oldest deadline 2024-04-25T11:55:09.408699+0000) 2024-04-25T11:55:14.782 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:14 smithi111 systemd[1]: Stopping Ceph osd.5 for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:55:15.074 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:14 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:14.832+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:15.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:14 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:14.879+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:15.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:14 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:14.879+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:15.075 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:14 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:14.792+0000 7f8c599de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:55:15.075 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:14 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:14.792+0000 7f8c599de700 -1 osd.5 102 *** Got signal Terminated *** 2024-04-25T11:55:15.076 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:14 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:14.792+0000 7f8c599de700 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:55:15.311 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:14 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:14.994+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:15.311 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:14 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:14.994+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:15.574 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:15 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:15.319+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:15.575 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:15 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:15.319+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:44.108661+0000 front 2024-04-25T11:54:44.108761+0000 (oldest deadline 2024-04-25T11:55:09.408699+0000) 2024-04-25T11:55:15.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:15 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:15.322+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:15.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:15 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:15.322+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:15.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:15 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:15.321+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:15.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:15 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:15.547+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:15.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:15 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:15.547+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:16.075 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:15 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:15.805+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:16.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:15 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:15.882+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:16.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:15 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:15.882+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:16.308 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:16 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:16.000+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:16.308 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:16 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:16.000+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:16.582 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:16 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:16.318+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:16.583 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:16 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:16.318+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:16.780 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:16 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:16.291+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:16.780 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:16 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:16.527+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:16.780 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:16 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:16.527+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:16.824 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:16 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:16.340+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:16.825 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:16 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:16.340+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:44.108661+0000 front 2024-04-25T11:54:44.108761+0000 (oldest deadline 2024-04-25T11:55:09.408699+0000) 2024-04-25T11:55:17.298 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:16 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:16.985+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:17.299 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:16 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:16.985+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:17.299 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:16 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:16.985+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:17.324 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:16 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:16.847+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:17.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:16 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:16.892+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:17.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:16 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:16.892+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:17.516 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:17 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:17.272+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:17.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:17 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:17.308+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:17.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:17 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:17.308+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:17.775 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:17 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:17.526+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:17.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:17 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:17.526+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:17.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:17 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:17.526+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:17.824 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:17 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:17.352+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:17.825 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:17 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:17.352+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:44.108661+0000 front 2024-04-25T11:54:44.108761+0000 (oldest deadline 2024-04-25T11:55:09.408699+0000) 2024-04-25T11:55:18.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:17 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:17.843+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:18.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:17 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:17.861+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:18.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:17 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:17.861+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:18.347 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:17 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:17.935+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:18.347 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:17 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:17.935+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:18.347 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:17 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:17.935+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:18.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:18 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:18.284+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:18.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:18 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:18.531+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:18.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:18 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:18.531+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:18.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:18 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:18.531+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:18.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:18 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:18.357+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:18.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:18 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:18.357+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:18.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:18 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:18.357+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.189668+0000 front 2024-04-25T11:54:52.189752+0000 (oldest deadline 2024-04-25T11:55:18.089766+0000) 2024-04-25T11:55:18.826 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:18 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:18.360+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:18.826 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:18 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:18.360+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:44.108661+0000 front 2024-04-25T11:54:44.108761+0000 (oldest deadline 2024-04-25T11:55:09.408699+0000) 2024-04-25T11:55:19.158 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:18 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:18.886+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:19.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:18 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:18.886+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:19.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:18 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:18.886+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:19.324 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:18 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:18.865+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:19.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:18 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:18.865+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:19.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:18 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:18.871+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:19.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:18 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:18.871+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:19.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:18 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:18.871+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:19.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:19 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:19.366+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:19.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:19 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:19.366+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:19.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:19 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:19.366+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.189668+0000 front 2024-04-25T11:54:52.189752+0000 (oldest deadline 2024-04-25T11:55:18.089766+0000) 2024-04-25T11:55:19.775 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:19 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:19.501+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:19.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:19 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:19.501+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:19.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:19 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:19.501+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:19.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:19 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:19.317+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:19.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:19 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:19.317+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:19.825 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:19 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:19.390+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.707673+0000 front 2024-04-25T11:54:37.707628+0000 (oldest deadline 2024-04-25T11:55:03.007656+0000) 2024-04-25T11:55:19.825 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:19 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5[44861]: 2024-04-25T11:55:19.390+0000 7f8c55fd2700 -1 osd.5 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:44.108661+0000 front 2024-04-25T11:54:44.108761+0000 (oldest deadline 2024-04-25T11:55:09.408699+0000) 2024-04-25T11:55:20.158 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:19 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:19.914+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:20.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:19 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:19.914+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:20.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:19 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:19.914+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:20.320 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:19 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:19.913+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:20.320 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:19 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:19.913+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:20.321 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:19 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:19.921+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:20.321 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:19 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:19.921+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:20.321 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:19 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:19.921+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:20.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:20 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:20.524+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:20.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:20 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:20.524+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:20.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:20 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:20.524+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:20.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:20 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:20.281+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:20.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:20 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:20.281+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:20.574 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:20 smithi111 podman[69662]: 2024-04-25 11:55:20.330891762 +0000 UTC m=+5.642519413 container died 9b336e1b8252c8f0225afb32402fd3b60ac4a744c43077cb96e24b4c1fbc8351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-04-25T11:55:20.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:20 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:20.374+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:20.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:20 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:20.374+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:20.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:20 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:20.374+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.189668+0000 front 2024-04-25T11:54:52.189752+0000 (oldest deadline 2024-04-25T11:55:18.089766+0000) 2024-04-25T11:55:20.910 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:20 smithi111 podman[69662]: 2024-04-25 11:55:20.735876514 +0000 UTC m=+6.047504209 container cleanup 9b336e1b8252c8f0225afb32402fd3b60ac4a744c43077cb96e24b4c1fbc8351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T11:55:20.910 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:20 smithi111 bash[69662]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5 2024-04-25T11:55:21.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:20 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:20.953+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:21.316 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:20 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:20.953+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:21.316 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:20 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:20.953+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:21.324 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:20 smithi111 podman[69679]: 2024-04-25 11:55:20.960124454 +0000 UTC m=+0.628250373 container remove 9b336e1b8252c8f0225afb32402fd3b60ac4a744c43077cb96e24b4c1fbc8351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-04-25T11:55:21.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:20 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:20.919+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:21.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:20 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:20.919+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:21.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:20 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:20.941+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:21.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:20 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:20.941+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:21.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:20 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:20.941+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:21.652 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:21 smithi111 podman[69770]: 2024-04-25 11:55:21.261734738 +0000 UTC m=+0.018304606 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:55:21.652 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:21 smithi111 podman[69770]: 2024-04-25 11:55:21.369246217 +0000 UTC m=+0.125816127 container create c75a004ef7e30be67c8d7b4b5422145cc688792207e7a2fac58addc9e023c189 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T11:55:21.653 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:21 smithi111 podman[69770]: 2024-04-25 11:55:21.579799902 +0000 UTC m=+0.336369769 container init c75a004ef7e30be67c8d7b4b5422145cc688792207e7a2fac58addc9e023c189 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:55:21.653 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:21 smithi111 podman[69770]: 2024-04-25 11:55:21.582758874 +0000 UTC m=+0.339328752 container start c75a004ef7e30be67c8d7b4b5422145cc688792207e7a2fac58addc9e023c189 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T11:55:21.653 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:21 smithi111 podman[69770]: 2024-04-25 11:55:21.663046005 +0000 UTC m=+0.419615873 container attach c75a004ef7e30be67c8d7b4b5422145cc688792207e7a2fac58addc9e023c189 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:55:21.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:21 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:21.326+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:21.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:21 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:21.326+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:21.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:21 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:21.326+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.189668+0000 front 2024-04-25T11:54:52.189752+0000 (oldest deadline 2024-04-25T11:55:18.089766+0000) 2024-04-25T11:55:21.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:21 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:21.489+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:21.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:21 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:21.489+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:21.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:21 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:21.489+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:21.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:21 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:21.288+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:21.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:21 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:21.288+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:21.921 INFO:journalctl@ceph.osd.5.smithi111.stdout:Apr 25 11:55:21 smithi111 podman[69770]: 2024-04-25 11:55:21.871734833 +0000 UTC m=+0.628304710 container died c75a004ef7e30be67c8d7b4b5422145cc688792207e7a2fac58addc9e023c189 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:55:21.922 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:21 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:21.906+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:21.922 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:21 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:21.906+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:21.922 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:21 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:21.931+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:21.923 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:21 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:21.931+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:21.923 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:21 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:21.931+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:22.273 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:21 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:21.974+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:22.273 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:21 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:21.974+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:22.273 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:21 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:21.974+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:22.397 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T11:55:22.435 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:55:22.435 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T11:55:22.435 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T11:55:22.436 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.6 2024-04-25T11:55:22.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:22 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:22.440+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:22.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:22 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:22.440+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:22.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:22 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:22.440+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:22.527 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:22 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:22.277+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:22.527 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:22 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:22.277+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:22.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:22 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:22.283+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:22.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:22 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:22.283+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:22.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:22 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:22.283+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.189668+0000 front 2024-04-25T11:54:52.189752+0000 (oldest deadline 2024-04-25T11:55:18.089766+0000) 2024-04-25T11:55:22.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:22 smithi083 systemd[1]: Stopping Ceph osd.6 for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:55:23.075 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:22 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:22.878+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:23.075 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:22 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:22.878+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:23.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:22 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:22.971+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:23.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:22 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:22.971+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:23.075 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:22 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:22.971+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:23.159 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:22 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:22.772+0000 7fbb3c401700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:55:23.159 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:22 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:22.772+0000 7fbb3c401700 -1 osd.6 102 *** Got signal Terminated *** 2024-04-25T11:55:23.159 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:22 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:22.772+0000 7fbb3c401700 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:55:23.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:22 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:22.931+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:23.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:22 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:22.931+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:23.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:22 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:22.931+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:23.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:23 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:23.301+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:23.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:23 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:23.301+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:23.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:23 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:23.301+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.189668+0000 front 2024-04-25T11:54:52.189752+0000 (oldest deadline 2024-04-25T11:55:18.089766+0000) 2024-04-25T11:55:23.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:23 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:23.322+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:23.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:23 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:23.322+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:23.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:23 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:23.427+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:23.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:23 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:23.427+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:23.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:23 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:23.427+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:24.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:23 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:23.920+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:24.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:23 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:23.920+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:24.315 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:23 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:23.920+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:24.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:23 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:23.899+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:24.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:23 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:23.899+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:24.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:24.010+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:24.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:24.010+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:24.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:24.010+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:24.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:24 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:24.325+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:24.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:24 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:24.325+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:24.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:24 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:24.325+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.189668+0000 front 2024-04-25T11:54:52.189752+0000 (oldest deadline 2024-04-25T11:55:18.089766+0000) 2024-04-25T11:55:24.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:24.355+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:24.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:24.355+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:24.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:24.377+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:24.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:24.377+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:24.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:24.377+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:25.291 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:24 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:24.960+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:25.291 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:24 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:24.960+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:25.291 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:24 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:24.960+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:25.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:24.923+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:25.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:24 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:24.923+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:25.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:25 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:25.021+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:25.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:25 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:25.021+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:25.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:25 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:25.021+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:25.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:25 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:25.301+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:25.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:25 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:25.301+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:25.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:25 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:25.301+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.189668+0000 front 2024-04-25T11:54:52.189752+0000 (oldest deadline 2024-04-25T11:55:18.089766+0000) 2024-04-25T11:55:25.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:25 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:25.301+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.590563+0000 front 2024-04-25T11:55:01.590613+0000 (oldest deadline 2024-04-25T11:55:25.090695+0000) 2024-04-25T11:55:25.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:25 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:25.345+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:25.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:25 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:25.345+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:25.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:25 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:25.424+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:25.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:25 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:25.424+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:25.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:25 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:25.424+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:25.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:25 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:25.424+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:26.268 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:25 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:25.934+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:26.269 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:25 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:25.934+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:26.269 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:25 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:25.934+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:26.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:25 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:25.912+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:26.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:25 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:25.912+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:26.326 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:25 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:25.912+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:26.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:25 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:25.998+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:26.327 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:25 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:25.998+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:26.327 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:25 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:25.998+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:26.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:26 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:26.279+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:26.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:26 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:26.279+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:26.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:26 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:26.279+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.189668+0000 front 2024-04-25T11:54:52.189752+0000 (oldest deadline 2024-04-25T11:55:18.089766+0000) 2024-04-25T11:55:26.660 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:26 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:26.279+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.590563+0000 front 2024-04-25T11:55:01.590613+0000 (oldest deadline 2024-04-25T11:55:25.090695+0000) 2024-04-25T11:55:26.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:26 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:26.303+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:26.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:26 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:26.303+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:26.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:26 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:26.457+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:26.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:26 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:26.457+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:26.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:26 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:26.457+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:26.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:26 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:26.457+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:27.269 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:26 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:26.953+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:27.269 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:26 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:26.953+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:27.269 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:26 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:26.953+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:27.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:26 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:26.903+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:27.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:26 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:26.903+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:27.326 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:26 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:26.903+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:27.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:27 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:27.031+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:27.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:27 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:27.031+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:27.327 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:27 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:27.031+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:27.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:27 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:27.274+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:27.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:27 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:27.274+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:27.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:27 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:27.441+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:27.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:27 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:27.441+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:27.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:27 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:27.441+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:27.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:27 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:27.441+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:27.658 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:27.279+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:37.087012+0000 front 2024-04-25T11:54:37.087412+0000 (oldest deadline 2024-04-25T11:55:04.087798+0000) 2024-04-25T11:55:27.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:27.279+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.489424+0000 front 2024-04-25T11:54:47.489603+0000 (oldest deadline 2024-04-25T11:55:12.189323+0000) 2024-04-25T11:55:27.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:27.279+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.189668+0000 front 2024-04-25T11:54:52.189752+0000 (oldest deadline 2024-04-25T11:55:18.089766+0000) 2024-04-25T11:55:27.659 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6[58025]: 2024-04-25T11:55:27.279+0000 7fbb349ed700 -1 osd.6 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.590563+0000 front 2024-04-25T11:55:01.590613+0000 (oldest deadline 2024-04-25T11:55:25.090695+0000) 2024-04-25T11:55:28.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:27.911+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:28.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:27.911+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:28.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:27 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:27.911+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:28.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:27 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:27.940+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:28.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:27 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:27.940+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:28.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:27 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:27.940+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:28.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:28 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:28.081+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:28.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:28 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:28.081+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:28.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:28 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:28.081+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:28.423 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:28 smithi083 podman[88413]: 2024-04-25 11:55:28.246711212 +0000 UTC m=+5.609457812 container died 9f33e0db3c6d8a4f10839f27bda9898d1d8bda92c3d42c6639464b45152bd3eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T11:55:28.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:28 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:28.274+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:28.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:28 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:28.274+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:28.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:28 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:28.483+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:28.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:28 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:28.483+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:28.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:28 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:28.483+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:28.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:28 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:28.483+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:28.695 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:28 smithi083 podman[88413]: 2024-04-25 11:55:28.622841981 +0000 UTC m=+5.985588569 container cleanup 9f33e0db3c6d8a4f10839f27bda9898d1d8bda92c3d42c6639464b45152bd3eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T11:55:28.695 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:28 smithi083 bash[88413]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6 2024-04-25T11:55:29.159 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:28 smithi083 podman[88431]: 2024-04-25 11:55:28.896178969 +0000 UTC m=+0.644528881 container remove 9f33e0db3c6d8a4f10839f27bda9898d1d8bda92c3d42c6639464b45152bd3eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T11:55:29.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:28 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:28.909+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:29.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:28 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:28.909+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:29.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:28 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:28.909+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:29.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:28 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:28.951+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:29.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:28 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:28.951+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:29.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:28 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:28.951+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:29.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:29 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:29.122+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:29.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:29 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:29.122+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:29.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:29 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:29.122+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:29.551 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:29 smithi083 podman[88531]: 2024-04-25 11:55:29.166503802 +0000 UTC m=+0.019411187 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:55:29.551 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:29 smithi083 podman[88531]: 2024-04-25 11:55:29.275212012 +0000 UTC m=+0.128119381 container create 83ccfe78158182cb09df38bbcf26cee8bbff3f89d6940cba9ec59034f3dfed45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T11:55:29.551 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:29 smithi083 podman[88531]: 2024-04-25 11:55:29.477461731 +0000 UTC m=+0.330369103 container init 83ccfe78158182cb09df38bbcf26cee8bbff3f89d6940cba9ec59034f3dfed45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T11:55:29.551 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:29 smithi083 podman[88531]: 2024-04-25 11:55:29.480615128 +0000 UTC m=+0.333522500 container start 83ccfe78158182cb09df38bbcf26cee8bbff3f89d6940cba9ec59034f3dfed45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T11:55:29.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:29 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:29.286+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:29.777 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:29 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:29.286+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:29.777 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:29 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:29.286+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:54:58.830340+0000 front 2024-04-25T11:54:58.830167+0000 (oldest deadline 2024-04-25T11:55:28.830732+0000) 2024-04-25T11:55:29.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:29 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:29.437+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:29.778 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:29 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:29.437+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:29.778 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:29 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:29.437+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:29.778 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:29 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:29.437+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:29.851 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:29 smithi083 podman[88531]: 2024-04-25 11:55:29.560810457 +0000 UTC m=+0.413717835 container attach 83ccfe78158182cb09df38bbcf26cee8bbff3f89d6940cba9ec59034f3dfed45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T11:55:29.851 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 25 11:55:29 smithi083 podman[88531]: 2024-04-25 11:55:29.793005624 +0000 UTC m=+0.645912993 container died 83ccfe78158182cb09df38bbcf26cee8bbff3f89d6940cba9ec59034f3dfed45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T11:55:30.158 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:29 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:29.861+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:30.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:29 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:29.861+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:30.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:29 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:29.861+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:30.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:29 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:29.861+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:30.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:29 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:29.994+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:30.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:29 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:29.994+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:30.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:29 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:29.994+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:30.325 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:30 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:30.074+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:30.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:30 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:30.074+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:30.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:30 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:30.074+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:30.383 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T11:55:30.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:55:30.423 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T11:55:30.423 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T11:55:30.424 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.7 2024-04-25T11:55:30.745 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:30 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:30.399+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:30.745 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:30 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:30.399+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:30.746 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:30 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:30.399+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:30.746 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:30 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:30.399+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:30.746 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:30 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:30.304+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:30.746 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:30 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:30.304+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:30.746 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:30 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:30.304+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:54:58.830340+0000 front 2024-04-25T11:54:58.830167+0000 (oldest deadline 2024-04-25T11:55:28.830732+0000) 2024-04-25T11:55:30.746 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:30 smithi094 systemd[1]: Stopping Ceph osd.7 for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:55:30.747 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:30 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:30.754+0000 7ff214b62700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:55:30.747 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:30 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:30.754+0000 7ff214b62700 -1 osd.7 102 *** Got signal Terminated *** 2024-04-25T11:55:30.747 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:30 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:30.754+0000 7ff214b62700 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:55:31.158 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:30 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:30.834+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:31.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:30 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:30.834+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:31.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:30 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:30.834+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:31.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:30 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:30.834+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:31.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:30 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:30.950+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:31.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:30 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:30.950+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:31.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:30 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:30.950+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:31.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:31 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:31.108+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:31.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:31 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:31.108+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:31.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:31 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:31.108+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:31.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:31 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:31.279+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:31.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:31 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:31.279+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:31.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:31 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:31.279+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:54:58.830340+0000 front 2024-04-25T11:54:58.830167+0000 (oldest deadline 2024-04-25T11:55:28.830732+0000) 2024-04-25T11:55:31.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:31 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:31.391+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:31.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:31 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:31.391+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:31.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:31 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:31.391+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:31.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:31 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:31.391+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:32.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:31 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:31.875+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:32.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:31 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:31.875+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:32.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:31 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:31.875+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:32.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:31 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:31.875+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:32.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:31 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:31.903+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:32.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:31 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:31.903+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:32.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:31 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:31.903+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:32.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:32 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:32.152+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:32.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:32 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:32.152+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:32.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:32 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:32.152+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:32.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:32 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:32.269+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:32.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:32 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:32.269+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:32.526 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:32 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:32.269+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:54:58.830340+0000 front 2024-04-25T11:54:58.830167+0000 (oldest deadline 2024-04-25T11:55:28.830732+0000) 2024-04-25T11:55:32.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:32 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:32.378+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:32.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:32 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:32.378+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:32.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:32 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:32.378+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:32.527 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:32 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:32.378+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:33.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:32 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:32.851+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:33.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:32 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:32.851+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:33.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:32 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:32.851+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:33.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:32 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:32.851+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:33.164 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:32 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:32.859+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:33.164 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:32 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:32.859+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:33.164 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:32 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:32.859+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:33.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:33 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:33.174+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:33.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:33 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:33.174+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:33.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:33 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:33.174+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:33.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:33.306+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:33.777 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:33.306+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:33.777 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:33.306+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:54:58.830340+0000 front 2024-04-25T11:54:58.830167+0000 (oldest deadline 2024-04-25T11:55:28.830732+0000) 2024-04-25T11:55:33.777 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:33.306+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.831132+0000 front 2024-04-25T11:55:08.831100+0000 (oldest deadline 2024-04-25T11:55:32.931297+0000) 2024-04-25T11:55:33.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:33.382+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:33.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:33.382+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:33.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:33.382+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:33.778 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:33.382+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:34.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:33 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:33.815+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:34.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:33 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:33.815+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:34.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:33 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:33.815+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:34.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:33 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:33.815+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:34.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:33 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:33.815+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:34.205 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:33 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:33.904+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:34.205 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:33 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:33.904+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:34.206 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:33 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:33.904+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:34.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:34 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:34.215+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:34.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:34 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:34.215+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:34.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:34 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:34.215+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:34.779 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:34 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:34.349+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:34.779 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:34 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:34.349+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:34.779 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:34 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:34.349+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:54:58.830340+0000 front 2024-04-25T11:54:58.830167+0000 (oldest deadline 2024-04-25T11:55:28.830732+0000) 2024-04-25T11:55:34.779 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:34 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:34.349+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.831132+0000 front 2024-04-25T11:55:08.831100+0000 (oldest deadline 2024-04-25T11:55:32.931297+0000) 2024-04-25T11:55:34.779 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:34 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:34.403+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:34.780 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:34 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:34.403+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:34.780 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:34 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:34.403+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:34.780 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:34 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:34.403+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:35.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:34 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:34.775+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:35.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:34 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:34.775+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:35.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:34 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:34.775+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:35.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:34 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:34.775+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:35.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:34 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:34.775+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:35.190 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:34 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:34.878+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:35.190 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:34 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:34.878+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:35.191 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:34 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:34.878+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:35.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:35 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:35.200+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:35.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:35 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:35.200+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:35.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:35 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:35.200+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:35.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:35 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:35.200+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:35.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:35 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:35.385+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:47.828244+0000 front 2024-04-25T11:54:47.828204+0000 (oldest deadline 2024-04-25T11:55:12.528356+0000) 2024-04-25T11:55:35.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:35 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:35.385+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:54.829253+0000 front 2024-04-25T11:54:54.829160+0000 (oldest deadline 2024-04-25T11:55:18.829580+0000) 2024-04-25T11:55:35.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:35 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:35.385+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:54:58.830340+0000 front 2024-04-25T11:54:58.830167+0000 (oldest deadline 2024-04-25T11:55:28.830732+0000) 2024-04-25T11:55:35.777 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:35 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7[51156]: 2024-04-25T11:55:35.385+0000 7ff20d14e700 -1 osd.7 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.831132+0000 front 2024-04-25T11:55:08.831100+0000 (oldest deadline 2024-04-25T11:55:32.931297+0000) 2024-04-25T11:55:35.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:35 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:35.396+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:35.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:35 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:35.396+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:35.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:35 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:35.396+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:35.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:35 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:35.396+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:36.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:35 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:35.789+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:36.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:35 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:35.789+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:36.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:35 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:35.789+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:36.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:35 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:35.789+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:36.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:35 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:35.789+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:36.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:35 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:35.919+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:36.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:35 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:35.919+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:36.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:35 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:35.919+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:36.326 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:35 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:35.919+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:07.473633+0000 front 2024-04-25T11:55:07.473699+0000 (oldest deadline 2024-04-25T11:55:35.673959+0000) 2024-04-25T11:55:36.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:36 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:36.155+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:36.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:36 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:36.155+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:36.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:36 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:36.155+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:36.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:36 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:36.155+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:36.462 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:36 smithi094 podman[69392]: 2024-04-25 11:55:36.202036309 +0000 UTC m=+5.574362204 container died fff1578a276b9b8da89ec60b5840534f5795301b38382c126ec9783ef8845d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:55:36.462 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:36 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:36.433+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:36.462 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:36 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:36.433+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:36.463 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:36 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:36.433+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:36.463 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:36 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:36.433+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:36.463 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:36 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:36.433+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:36.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:36 smithi094 podman[69392]: 2024-04-25 11:55:36.571558495 +0000 UTC m=+5.943884398 container cleanup fff1578a276b9b8da89ec60b5840534f5795301b38382c126ec9783ef8845d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T11:55:36.776 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:36 smithi094 bash[69392]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7 2024-04-25T11:55:37.114 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:36 smithi094 podman[69409]: 2024-04-25 11:55:36.814337261 +0000 UTC m=+0.613536209 container remove fff1578a276b9b8da89ec60b5840534f5795301b38382c126ec9783ef8845d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T11:55:37.261 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:36 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:36.749+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:37.261 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:36 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:36.749+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:37.261 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:36 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:36.749+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:37.261 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:36 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:36.749+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:37.261 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:36 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:36.749+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:37.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:36 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:36.941+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:37.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:36 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:36.941+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:37.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:36 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:36.941+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:37.326 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:36 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:36.941+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:07.473633+0000 front 2024-04-25T11:55:07.473699+0000 (oldest deadline 2024-04-25T11:55:35.673959+0000) 2024-04-25T11:55:37.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:37 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:37.163+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:37.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:37 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:37.163+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:37.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:37 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:37.163+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:37.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:37 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:37.163+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:37.395 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:37 smithi094 podman[69498]: 2024-04-25 11:55:37.027338214 +0000 UTC m=+0.017989188 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:55:37.395 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:37 smithi094 podman[69498]: 2024-04-25 11:55:37.148391514 +0000 UTC m=+0.139042476 container create 63cb2897df15985cb4ba3785939a19b37820ce703d3c2bcbea26210711e82da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-25T11:55:37.689 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:37 smithi094 podman[69498]: 2024-04-25 11:55:37.408839153 +0000 UTC m=+0.399490115 container init 63cb2897df15985cb4ba3785939a19b37820ce703d3c2bcbea26210711e82da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T11:55:37.689 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:37 smithi094 podman[69498]: 2024-04-25 11:55:37.411846514 +0000 UTC m=+0.402497488 container start 63cb2897df15985cb4ba3785939a19b37820ce703d3c2bcbea26210711e82da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T11:55:37.689 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:37 smithi094 podman[69498]: 2024-04-25 11:55:37.500639558 +0000 UTC m=+0.491290519 container attach 63cb2897df15985cb4ba3785939a19b37820ce703d3c2bcbea26210711e82da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T11:55:37.690 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:37 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:37.405+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:37.690 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:37 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:37.405+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:37.690 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:37 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:37.405+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:37.690 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:37 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:37.405+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:37.690 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:37 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:37.405+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:37.993 INFO:journalctl@ceph.osd.7.smithi094.stdout:Apr 25 11:55:37 smithi094 podman[69498]: 2024-04-25 11:55:37.699768322 +0000 UTC m=+0.690419284 container died 63cb2897df15985cb4ba3785939a19b37820ce703d3c2bcbea26210711e82da5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:55:38.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:37 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:37.729+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:38.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:37 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:37.729+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:38.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:37 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:37.729+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:38.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:37 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:37.729+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:38.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:37 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:37.729+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:38.294 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T11:55:38.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:37 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:37.930+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:38.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:37 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:37.930+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:38.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:37 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:37.930+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:38.326 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:37 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:37.930+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:07.473633+0000 front 2024-04-25T11:55:07.473699+0000 (oldest deadline 2024-04-25T11:55:35.673959+0000) 2024-04-25T11:55:38.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:38.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:38.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:38.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:38.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:38.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:38.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:38.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:38.332 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:55:38.332 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T11:55:38.333 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T11:55:38.333 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.8 2024-04-25T11:55:38.642 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:38 smithi111 systemd[1]: Stopping Ceph osd.8 for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:55:38.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:38 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:38.413+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:38.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:38 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:38.413+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:38.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:38 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:38.413+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:38.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:38 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:38.413+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:38.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:38 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:38.413+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:39.075 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:38.651+0000 7f5699895700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:55:39.075 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:38.651+0000 7f5699895700 -1 osd.8 102 *** Got signal Terminated *** 2024-04-25T11:55:39.075 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:38.651+0000 7f5699895700 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:55:39.075 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:38.885+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:39.075 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:38.885+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:39.076 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:38.885+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:39.076 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:38 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:38.885+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:07.473633+0000 front 2024-04-25T11:55:07.473699+0000 (oldest deadline 2024-04-25T11:55:35.673959+0000) 2024-04-25T11:55:39.157 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:38 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:38.770+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:39.157 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:38 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:38.770+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:39.158 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:38 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:38.770+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:39.158 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:38 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:38.770+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:39.158 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:38 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:38.770+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:39.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:39 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:39.125+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:39.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:39 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:39.125+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:39.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:39 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:39.125+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:39.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:39 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:39.125+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:39.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:39 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:39.384+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:39.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:39 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:39.384+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:39.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:39 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:39.384+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:39.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:39 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:39.384+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:39.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:39 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:39.384+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:40.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:39 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:39.762+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:40.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:39 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:39.762+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:40.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:39 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:39.762+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:40.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:39 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:39.762+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:40.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:39 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:39.762+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:40.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:39 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:39.935+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:40.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:39 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:39.935+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:40.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:39 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:39.935+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:40.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:39 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:39.935+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:07.473633+0000 front 2024-04-25T11:55:07.473699+0000 (oldest deadline 2024-04-25T11:55:35.673959+0000) 2024-04-25T11:55:40.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:40 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:40.152+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:40.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:40 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:40.152+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:40.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:40 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:40.152+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:40.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:40 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:40.152+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:40.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:40 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:40.425+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:40.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:40 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:40.425+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:40.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:40 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:40.425+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:40.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:40 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:40.425+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:40.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:40 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:40.425+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:41.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:40.809+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:41.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:40.809+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:41.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:40.809+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:41.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:40.809+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:41.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:40 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:40.809+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:41.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:40 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:40.896+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:41.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:40 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:40.896+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:41.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:40 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:40.896+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:41.326 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:40 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:40.896+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:07.473633+0000 front 2024-04-25T11:55:07.473699+0000 (oldest deadline 2024-04-25T11:55:35.673959+0000) 2024-04-25T11:55:41.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:41 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:41.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:41.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:41 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:41.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:41.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:41 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:41.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:41.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:41 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:41.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:41.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:41 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:41.457+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:41.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:41 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:41.457+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:41.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:41 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:41.457+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:41.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:41 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:41.457+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:41.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:41 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:41.457+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:42.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:41 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:41.784+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:42.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:41 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:41.784+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:42.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:41 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:41.784+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:42.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:41 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:41.784+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:42.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:41 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:41.784+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:42.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:41 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:41.784+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:18.883578+0000 front 2024-04-25T11:55:18.883678+0000 (oldest deadline 2024-04-25T11:55:41.783541+0000) 2024-04-25T11:55:42.226 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:41 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:41.932+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:42.227 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:41 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:41.932+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:42.227 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:41 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:41.932+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:42.227 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:41 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:41.932+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:07.473633+0000 front 2024-04-25T11:55:07.473699+0000 (oldest deadline 2024-04-25T11:55:35.673959+0000) 2024-04-25T11:55:42.227 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:42 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:42.165+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:42.227 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:42 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:42.165+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:42.228 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:42 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:42.165+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:42.228 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:42 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:42.165+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:42.228 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:42 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:42.165+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:42.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:42 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:42.484+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:42.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:42 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:42.484+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:42.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:42 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:42.484+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:42.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:42 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:42.484+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:42.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:42 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:42.484+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:43.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:42 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:42.820+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:43.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:42 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:42.820+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:43.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:42 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:42.820+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:43.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:42 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:42.820+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:43.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:42 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:42.820+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:43.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:42 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:42.820+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:18.883578+0000 front 2024-04-25T11:55:18.883678+0000 (oldest deadline 2024-04-25T11:55:41.783541+0000) 2024-04-25T11:55:43.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:42 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:42.962+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:42.469360+0000 front 2024-04-25T11:54:42.469328+0000 (oldest deadline 2024-04-25T11:55:08.369365+0000) 2024-04-25T11:55:43.325 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:42 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:42.962+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:53.471533+0000 front 2024-04-25T11:54:53.471352+0000 (oldest deadline 2024-04-25T11:55:18.171489+0000) 2024-04-25T11:55:43.326 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:42 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:42.962+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.072517+0000 front 2024-04-25T11:55:01.072599+0000 (oldest deadline 2024-04-25T11:55:25.772567+0000) 2024-04-25T11:55:43.326 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:42 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8[51465]: 2024-04-25T11:55:42.962+0000 7f5695e89700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:07.473633+0000 front 2024-04-25T11:55:07.473699+0000 (oldest deadline 2024-04-25T11:55:35.673959+0000) 2024-04-25T11:55:43.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:43 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:43.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:43.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:43 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:43.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:43.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:43 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:43.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:43.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:43 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:43.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:43.327 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:43 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:43.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:43.778 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:43 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:43.530+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:43.778 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:43 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:43.530+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:43.779 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:43 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:43.530+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:43.779 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:43 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:43.530+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:43.779 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:43 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:43.530+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:44.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:43 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:43.813+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:44.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:43 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:43.813+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:44.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:43 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:43.813+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:44.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:43 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:43.813+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:44.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:43 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:43.813+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:44.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:43 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:43.813+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:18.883578+0000 front 2024-04-25T11:55:18.883678+0000 (oldest deadline 2024-04-25T11:55:41.783541+0000) 2024-04-25T11:55:44.259 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:44 smithi111 podman[69878]: 2024-04-25 11:55:44.006179135 +0000 UTC m=+5.476578609 container died 3df1d899de3a12ba2c43adac15fe381bd5de69aadd577a95b1b06f769c255e9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T11:55:44.260 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:44 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:44.081+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:44.260 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:44 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:44.081+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:44.260 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:44 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:44.081+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:44.260 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:44 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:44.081+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:44.260 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:44 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:44.081+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:44.575 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:44 smithi111 podman[69878]: 2024-04-25 11:55:44.368526991 +0000 UTC m=+5.838926470 container cleanup 3df1d899de3a12ba2c43adac15fe381bd5de69aadd577a95b1b06f769c255e9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T11:55:44.575 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:44 smithi111 bash[69878]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8 2024-04-25T11:55:44.947 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:44 smithi111 podman[69907]: 2024-04-25 11:55:44.645152678 +0000 UTC m=+0.633850214 container remove 3df1d899de3a12ba2c43adac15fe381bd5de69aadd577a95b1b06f769c255e9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T11:55:45.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:44 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:44.568+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:45.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:44 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:44.568+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:45.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:44 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:44.568+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:45.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:44 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:44.568+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:45.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:44 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:44.568+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:45.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:44 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:44.849+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:45.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:44 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:44.849+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:45.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:44 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:44.849+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:45.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:44 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:44.849+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:45.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:44 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:44.849+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:45.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:44 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:44.849+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:18.883578+0000 front 2024-04-25T11:55:18.883678+0000 (oldest deadline 2024-04-25T11:55:41.783541+0000) 2024-04-25T11:55:45.199 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:44 smithi111 podman[69998]: 2024-04-25 11:55:44.859784764 +0000 UTC m=+0.017971078 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:55:45.200 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:44 smithi111 podman[69998]: 2024-04-25 11:55:44.970821155 +0000 UTC m=+0.129007496 container create 7abea08fdd1dbc9d1870a00b71011c061a570c1f9c74acd9b408c020a599934f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:55:45.200 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:45 smithi111 podman[69998]: 2024-04-25 11:55:45.206863743 +0000 UTC m=+0.365050042 container init 7abea08fdd1dbc9d1870a00b71011c061a570c1f9c74acd9b408c020a599934f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T11:55:45.200 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:45 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:45.117+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:45.200 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:45 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:45.117+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:45.200 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:45 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:45.117+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:45.201 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:45 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:45.117+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:45.201 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:45 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:45.117+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:45.488 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:45 smithi111 podman[69998]: 2024-04-25 11:55:45.2098493 +0000 UTC m=+0.368035600 container start 7abea08fdd1dbc9d1870a00b71011c061a570c1f9c74acd9b408c020a599934f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T11:55:45.488 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:45 smithi111 podman[69998]: 2024-04-25 11:55:45.348612753 +0000 UTC m=+0.506799063 container attach 7abea08fdd1dbc9d1870a00b71011c061a570c1f9c74acd9b408c020a599934f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T11:55:45.800 INFO:journalctl@ceph.osd.8.smithi111.stdout:Apr 25 11:55:45 smithi111 podman[69998]: 2024-04-25 11:55:45.498680348 +0000 UTC m=+0.656866653 container died 7abea08fdd1dbc9d1870a00b71011c061a570c1f9c74acd9b408c020a599934f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:55:46.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:45 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:45.563+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:46.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:45 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:45.563+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:46.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:45 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:45.563+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:46.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:45 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:45.563+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:46.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:45 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:45.563+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:46.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:45 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:45.563+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.051967+0000 front 2024-04-25T11:55:19.051946+0000 (oldest deadline 2024-04-25T11:55:44.951772+0000) 2024-04-25T11:55:46.105 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T11:55:46.142 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:55:46.142 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T11:55:46.143 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T11:55:46.143 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.9 2024-04-25T11:55:46.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:45 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:45.808+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:46.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:45 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:45.808+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:46.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:45 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:45.808+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:46.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:45 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:45.808+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:46.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:45 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:45.808+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:46.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:45 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:45.808+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:18.883578+0000 front 2024-04-25T11:55:18.883678+0000 (oldest deadline 2024-04-25T11:55:41.783541+0000) 2024-04-25T11:55:46.432 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:46 smithi083 systemd[1]: Stopping Ceph osd.9 for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:55:46.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:46 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:46.162+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:46.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:46 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:46.162+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:46.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:46 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:46.162+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:46.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:46 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:46.162+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:46.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:46 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:46.162+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:46.788 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:46 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:46.442+0000 7fd322166700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:55:46.789 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:46 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:46.442+0000 7fd322166700 -1 osd.9 102 *** Got signal Terminated *** 2024-04-25T11:55:46.789 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:46 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:46.442+0000 7fd322166700 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:55:46.789 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:46 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:46.798+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:46.789 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:46 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:46.798+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:46.789 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:46 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:46.798+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:46.789 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:46 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:46.798+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:47.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:46 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:46.541+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:47.026 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:46 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:46.541+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:47.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:46 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:46.542+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:47.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:46 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:46.542+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:47.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:46 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:46.542+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:47.027 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:46 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:46.542+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.051967+0000 front 2024-04-25T11:55:19.051946+0000 (oldest deadline 2024-04-25T11:55:44.951772+0000) 2024-04-25T11:55:47.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:46 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:46.798+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:47.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:46 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:46.798+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:18.883578+0000 front 2024-04-25T11:55:18.883678+0000 (oldest deadline 2024-04-25T11:55:41.783541+0000) 2024-04-25T11:55:47.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:47 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:47.208+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:47.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:47 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:47.208+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:47.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:47 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:47.208+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:47.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:47 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:47.208+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:47.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:47 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:47.208+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:47.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:47 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:47.506+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:47.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:47 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:47.506+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:47.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:47 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:47.506+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:47.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:47 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:47.506+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:47.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:47 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:47.506+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:47.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:47 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:47.506+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.051967+0000 front 2024-04-25T11:55:19.051946+0000 (oldest deadline 2024-04-25T11:55:44.951772+0000) 2024-04-25T11:55:48.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:47 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:47.793+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:48.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:47 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:47.793+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:48.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:47 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:47.793+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:48.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:47 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:47.793+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:48.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:47 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:47.793+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:48.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:47 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:47.793+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:18.883578+0000 front 2024-04-25T11:55:18.883678+0000 (oldest deadline 2024-04-25T11:55:41.783541+0000) 2024-04-25T11:55:48.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:48 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:48.186+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:48.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:48 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:48.186+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:48.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:48 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:48.186+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:48.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:48 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:48.186+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:48.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:48 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:48.186+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:48.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:48 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:48.490+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:48.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:48 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:48.490+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:48.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:48 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:48.490+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:48.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:48 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:48.490+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:48.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:48 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:48.490+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:48.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:48 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:48.490+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.051967+0000 front 2024-04-25T11:55:19.051946+0000 (oldest deadline 2024-04-25T11:55:44.951772+0000) 2024-04-25T11:55:49.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:48 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:48.748+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:49.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:48 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:48.748+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:49.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:48 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:48.748+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:49.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:48 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:48.748+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:49.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:48 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:48.748+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:49.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:48 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:48.748+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:18.883578+0000 front 2024-04-25T11:55:18.883678+0000 (oldest deadline 2024-04-25T11:55:41.783541+0000) 2024-04-25T11:55:49.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:49 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:49.200+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:49.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:49 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:49.200+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:49.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:49 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:49.200+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:49.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:49 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:49.200+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:49.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:49 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:49.200+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:49.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:49 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:49.513+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:49.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:49 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:49.513+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:49.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:49 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:49.513+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:49.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:49 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:49.513+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:49.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:49 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:49.513+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:49.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:49 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:49.513+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.051967+0000 front 2024-04-25T11:55:19.051946+0000 (oldest deadline 2024-04-25T11:55:44.951772+0000) 2024-04-25T11:55:50.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:49 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:49.737+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:50.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:49 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:49.737+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:50.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:49 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:49.737+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:50.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:49 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:49.737+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:50.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:49 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:49.737+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:50.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:49 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:49.737+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:18.883578+0000 front 2024-04-25T11:55:18.883678+0000 (oldest deadline 2024-04-25T11:55:41.783541+0000) 2024-04-25T11:55:50.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:50 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:50.209+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:50.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:50 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:50.209+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:50.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:50 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:50.209+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:50.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:50 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:50.209+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:50.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:50 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:50.209+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:50.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:50 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:50.477+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:50.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:50 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:50.477+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:50.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:50 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:50.477+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:50.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:50 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:50.477+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:50.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:50 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:50.477+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:50.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:50 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:50.477+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.051967+0000 front 2024-04-25T11:55:19.051946+0000 (oldest deadline 2024-04-25T11:55:44.951772+0000) 2024-04-25T11:55:50.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:50 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:50.477+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:26.052747+0000 front 2024-04-25T11:55:26.052763+0000 (oldest deadline 2024-04-25T11:55:50.152630+0000) 2024-04-25T11:55:51.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:50 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:50.691+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:35.077423+0000 front 2024-04-25T11:54:35.077501+0000 (oldest deadline 2024-04-25T11:55:00.377643+0000) 2024-04-25T11:55:51.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:50 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:50.691+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.678671+0000 front 2024-04-25T11:54:45.678766+0000 (oldest deadline 2024-04-25T11:55:09.778545+0000) 2024-04-25T11:55:51.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:50 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:50.691+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.679777+0000 front 2024-04-25T11:54:52.679750+0000 (oldest deadline 2024-04-25T11:55:16.779552+0000) 2024-04-25T11:55:51.159 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:50 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:50.691+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:03.681419+0000 front 2024-04-25T11:55:03.681465+0000 (oldest deadline 2024-04-25T11:55:28.981450+0000) 2024-04-25T11:55:51.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:50 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:50.691+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:08.981998+0000 front 2024-04-25T11:55:08.982095+0000 (oldest deadline 2024-04-25T11:55:33.681929+0000) 2024-04-25T11:55:51.160 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:50 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9[65052]: 2024-04-25T11:55:50.692+0000 7fd31a752700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:18.883578+0000 front 2024-04-25T11:55:18.883678+0000 (oldest deadline 2024-04-25T11:55:41.783541+0000) 2024-04-25T11:55:51.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:51 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:51.202+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:51.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:51 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:51.202+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:51.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:51 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:51.202+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:51.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:51 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:51.202+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:51.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:51 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:51.202+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:51.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:51 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:51.202+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:55:51.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:51 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:51.490+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:51.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:51 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:51.490+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:51.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:51 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:51.490+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:51.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:51 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:51.490+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:51.778 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:51 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:51.490+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:51.778 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:51 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:51.490+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.051967+0000 front 2024-04-25T11:55:19.051946+0000 (oldest deadline 2024-04-25T11:55:44.951772+0000) 2024-04-25T11:55:51.778 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:51 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:51.490+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:26.052747+0000 front 2024-04-25T11:55:26.052763+0000 (oldest deadline 2024-04-25T11:55:50.152630+0000) 2024-04-25T11:55:52.107 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:51 smithi083 podman[88637]: 2024-04-25 11:55:51.848165846 +0000 UTC m=+5.503751406 container died 321290513c17b91ec5c755361693fdd0b22192ae171cc56f568e71eb8624b8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T11:55:52.409 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:52 smithi083 podman[88637]: 2024-04-25 11:55:52.200058199 +0000 UTC m=+5.855643758 container cleanup 321290513c17b91ec5c755361693fdd0b22192ae171cc56f568e71eb8624b8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T11:55:52.409 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:52 smithi083 bash[88637]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9 2024-04-25T11:55:52.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:52 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:52.216+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:52.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:52 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:52.216+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:52.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:52 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:52.216+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:52.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:52 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:52.216+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:52.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:52 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:52.216+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:52.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:52 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:52.216+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:55:52.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:52 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:52.487+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:52.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:52 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:52.487+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:52.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:52 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:52.487+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:52.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:52 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:52.487+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:52.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:52 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:52.487+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:52.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:52 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:52.487+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.051967+0000 front 2024-04-25T11:55:19.051946+0000 (oldest deadline 2024-04-25T11:55:44.951772+0000) 2024-04-25T11:55:52.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:52 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:52.487+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:26.052747+0000 front 2024-04-25T11:55:26.052763+0000 (oldest deadline 2024-04-25T11:55:50.152630+0000) 2024-04-25T11:55:52.801 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:52 smithi083 podman[88654]: 2024-04-25 11:55:52.484582464 +0000 UTC m=+0.633835719 container remove 321290513c17b91ec5c755361693fdd0b22192ae171cc56f568e71eb8624b8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T11:55:53.119 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:52 smithi083 podman[88747]: 2024-04-25 11:55:52.714985386 +0000 UTC m=+0.017738330 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:55:53.120 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:52 smithi083 podman[88747]: 2024-04-25 11:55:52.827264206 +0000 UTC m=+0.130017148 container create b0a1cd237a14f7f173b572e74511392843790d9e18ac53a6321f77045f87d28d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T11:55:53.120 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:53 smithi083 podman[88747]: 2024-04-25 11:55:53.054550177 +0000 UTC m=+0.357303120 container init b0a1cd237a14f7f173b572e74511392843790d9e18ac53a6321f77045f87d28d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-25T11:55:53.120 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:53 smithi083 podman[88747]: 2024-04-25 11:55:53.057820286 +0000 UTC m=+0.360573240 container start b0a1cd237a14f7f173b572e74511392843790d9e18ac53a6321f77045f87d28d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:55:53.120 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:53 smithi083 podman[88747]: 2024-04-25 11:55:53.129452944 +0000 UTC m=+0.432205903 container attach b0a1cd237a14f7f173b572e74511392843790d9e18ac53a6321f77045f87d28d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:55:53.409 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 25 11:55:53 smithi083 podman[88747]: 2024-04-25 11:55:53.364494366 +0000 UTC m=+0.667247308 container died b0a1cd237a14f7f173b572e74511392843790d9e18ac53a6321f77045f87d28d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T11:55:53.524 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:53 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:53.254+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:53.525 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:53 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:53.254+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:53.525 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:53 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:53.254+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:53.525 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:53 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:53.254+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:53.525 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:53 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:53.254+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:53.525 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:53 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:53.254+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:55:53.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:53 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:53.448+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:53.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:53 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:53.448+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:53.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:53 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:53.448+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:53.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:53 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:53.448+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:53.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:53 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:53.448+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:53.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:53 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:53.448+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.051967+0000 front 2024-04-25T11:55:19.051946+0000 (oldest deadline 2024-04-25T11:55:44.951772+0000) 2024-04-25T11:55:53.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:53 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:53.448+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:26.052747+0000 front 2024-04-25T11:55:26.052763+0000 (oldest deadline 2024-04-25T11:55:50.152630+0000) 2024-04-25T11:55:53.925 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T11:55:54.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:55:54.003 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T11:55:54.003 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T11:55:54.004 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.10 2024-04-25T11:55:54.296 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:54 smithi094 systemd[1]: Stopping Ceph osd.10 for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:55:54.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:54 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:54.233+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:54.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:54 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:54.233+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:54.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:54 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:54.233+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:54.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:54 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:54.233+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:54.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:54 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:54.233+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:54.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:54 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:54.233+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:55:54.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:54 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:54.305+0000 7f3389fdf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:55:54.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:54 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:54.305+0000 7f3389fdf700 -1 osd.10 102 *** Got signal Terminated *** 2024-04-25T11:55:54.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:54 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:54.305+0000 7f3389fdf700 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:55:54.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:54 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:54.425+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:54.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:54 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:54.425+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:54.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:54 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:54.425+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:54.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:54 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:54.425+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:54.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:54 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:54.425+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:54.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:54 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:54.425+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.051967+0000 front 2024-04-25T11:55:19.051946+0000 (oldest deadline 2024-04-25T11:55:44.951772+0000) 2024-04-25T11:55:54.778 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:54 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:54.425+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:26.052747+0000 front 2024-04-25T11:55:26.052763+0000 (oldest deadline 2024-04-25T11:55:50.152630+0000) 2024-04-25T11:55:55.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:55 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:55.223+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:55.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:55 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:55.223+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:55.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:55 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:55.223+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:55.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:55 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:55.223+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:55.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:55 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:55.223+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:55.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:55 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:55.223+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:55:55.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:55 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:55.423+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:55.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:55 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:55.423+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:55.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:55 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:55.423+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:55.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:55 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:55.423+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:55.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:55 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:55.423+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:55.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:55 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:55.423+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.051967+0000 front 2024-04-25T11:55:19.051946+0000 (oldest deadline 2024-04-25T11:55:44.951772+0000) 2024-04-25T11:55:55.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:55 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:55.423+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:26.052747+0000 front 2024-04-25T11:55:26.052763+0000 (oldest deadline 2024-04-25T11:55:50.152630+0000) 2024-04-25T11:55:56.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:56 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:56.197+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:56.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:56 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:56.197+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:56.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:56 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:56.197+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:56.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:56 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:56.197+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:56.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:56 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:56.197+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:56.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:56 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:56.197+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:55:56.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:56 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:56.430+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:56.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:56 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:56.430+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:56.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:56 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:56.430+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:56.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:56 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:56.430+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:56.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:56 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:56.430+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:56.778 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:56 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:56.430+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.051967+0000 front 2024-04-25T11:55:19.051946+0000 (oldest deadline 2024-04-25T11:55:44.951772+0000) 2024-04-25T11:55:56.778 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:56 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:56.430+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:26.052747+0000 front 2024-04-25T11:55:26.052763+0000 (oldest deadline 2024-04-25T11:55:50.152630+0000) 2024-04-25T11:55:57.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:57 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:57.177+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:57.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:57 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:57.177+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:57.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:57 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:57.177+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:57.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:57 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:57.177+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:57.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:57 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:57.177+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:57.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:57 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:57.177+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:55:57.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:57 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:57.177+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-25T11:55:30.830973+0000 front 2024-04-25T11:55:30.830953+0000 (oldest deadline 2024-04-25T11:55:56.730926+0000) 2024-04-25T11:55:57.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:57 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:57.466+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:57.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:57 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:57.466+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:57.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:57 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:57.466+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:57.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:57 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:57.466+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:57.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:57 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:57.466+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:57.778 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:57 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:57.466+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.051967+0000 front 2024-04-25T11:55:19.051946+0000 (oldest deadline 2024-04-25T11:55:44.951772+0000) 2024-04-25T11:55:57.778 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:57 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:57.466+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:26.052747+0000 front 2024-04-25T11:55:26.052763+0000 (oldest deadline 2024-04-25T11:55:50.152630+0000) 2024-04-25T11:55:58.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:58 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:58.173+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:58.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:58 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:58.173+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:58.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:58 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:58.173+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:58.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:58 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:58.173+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:58.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:58 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:58.173+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:58.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:58 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:58.173+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:55:58.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:58 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:58.173+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-25T11:55:30.830973+0000 front 2024-04-25T11:55:30.830953+0000 (oldest deadline 2024-04-25T11:55:56.730926+0000) 2024-04-25T11:55:58.776 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:58 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:58.473+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.342848+0000 front 2024-04-25T11:54:39.342862+0000 (oldest deadline 2024-04-25T11:55:03.442634+0000) 2024-04-25T11:55:58.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:58 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:58.473+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:43.443021+0000 front 2024-04-25T11:54:43.442977+0000 (oldest deadline 2024-04-25T11:55:08.743143+0000) 2024-04-25T11:55:58.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:58 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:58.473+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:52.144857+0000 front 2024-04-25T11:54:52.145046+0000 (oldest deadline 2024-04-25T11:55:17.444989+0000) 2024-04-25T11:55:58.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:58 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:58.473+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6814 osd.3 since back 2024-04-25T11:55:01.847926+0000 front 2024-04-25T11:55:01.848076+0000 (oldest deadline 2024-04-25T11:55:24.747796+0000) 2024-04-25T11:55:58.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:58 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:58.473+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:09.849783+0000 front 2024-04-25T11:55:09.849730+0000 (oldest deadline 2024-04-25T11:55:35.749824+0000) 2024-04-25T11:55:58.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:58 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:58.473+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.051967+0000 front 2024-04-25T11:55:19.051946+0000 (oldest deadline 2024-04-25T11:55:44.951772+0000) 2024-04-25T11:55:58.777 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:58 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10[57743]: 2024-04-25T11:55:58.473+0000 7f33825cb700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:26.052747+0000 front 2024-04-25T11:55:26.052763+0000 (oldest deadline 2024-04-25T11:55:50.152630+0000) 2024-04-25T11:55:59.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:59 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:59.143+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:55:59.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:59 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:59.143+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:55:59.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:59 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:59.143+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:55:59.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:59 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:59.143+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:55:59.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:59 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:59.143+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:55:59.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:59 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:59.143+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:55:59.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:55:59 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:55:59.143+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-25T11:55:30.830973+0000 front 2024-04-25T11:55:30.830953+0000 (oldest deadline 2024-04-25T11:55:56.730926+0000) 2024-04-25T11:55:59.955 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:55:59 smithi094 podman[69616]: 2024-04-25 11:55:59.707611776 +0000 UTC m=+5.502894993 container died b26b8291c60a8170cad041f4c1731d8059fbbccbbe713077c92ca26f5500ea87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T11:56:00.264 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:56:00 smithi094 podman[69616]: 2024-04-25 11:56:00.055760445 +0000 UTC m=+5.851043663 container cleanup b26b8291c60a8170cad041f4c1731d8059fbbccbbe713077c92ca26f5500ea87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:56:00.264 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:56:00 smithi094 bash[69616]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10 2024-04-25T11:56:00.526 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:56:00 smithi094 podman[69634]: 2024-04-25 11:56:00.27368352 +0000 UTC m=+0.556507073 container remove b26b8291c60a8170cad041f4c1731d8059fbbccbbe713077c92ca26f5500ea87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-25T11:56:00.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:00 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:00.142+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:56:00.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:00 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:00.142+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:56:00.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:00 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:00.142+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:56:00.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:00 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:00.142+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:56:00.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:00 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:00.142+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:56:00.577 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:00 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:00.142+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:56:00.577 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:00 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:00.142+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-25T11:55:30.830973+0000 front 2024-04-25T11:55:30.830953+0000 (oldest deadline 2024-04-25T11:55:56.730926+0000) 2024-04-25T11:56:00.958 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:56:00 smithi094 podman[69725]: 2024-04-25 11:56:00.572865545 +0000 UTC m=+0.018188628 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:56:00.958 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:56:00 smithi094 podman[69725]: 2024-04-25 11:56:00.674279474 +0000 UTC m=+0.119602539 container create a4083e5e18c31d026e0b2a23cf54f85ef4ffdafd230e2f20c36d469f84f8ece0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T11:56:00.959 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:56:00 smithi094 podman[69725]: 2024-04-25 11:56:00.89346149 +0000 UTC m=+0.338784557 container init a4083e5e18c31d026e0b2a23cf54f85ef4ffdafd230e2f20c36d469f84f8ece0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T11:56:00.959 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:56:00 smithi094 podman[69725]: 2024-04-25 11:56:00.896974817 +0000 UTC m=+0.342297886 container start a4083e5e18c31d026e0b2a23cf54f85ef4ffdafd230e2f20c36d469f84f8ece0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-04-25T11:56:00.959 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:56:00 smithi094 podman[69725]: 2024-04-25 11:56:00.968355229 +0000 UTC m=+0.413678303 container attach a4083e5e18c31d026e0b2a23cf54f85ef4ffdafd230e2f20c36d469f84f8ece0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:56:01.277 INFO:journalctl@ceph.osd.10.smithi094.stdout:Apr 25 11:56:01 smithi094 podman[69725]: 2024-04-25 11:56:01.185259071 +0000 UTC m=+0.630582138 container died a4083e5e18c31d026e0b2a23cf54f85ef4ffdafd230e2f20c36d469f84f8ece0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T11:56:01.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:01 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:01.098+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:56:01.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:01 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:01.098+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:56:01.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:01 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:01.098+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:56:01.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:01 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:01.098+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:56:01.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:01 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:01.098+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:56:01.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:01 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:01.098+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:56:01.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:01 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:01.098+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-25T11:55:30.830973+0000 front 2024-04-25T11:55:30.830953+0000 (oldest deadline 2024-04-25T11:55:56.730926+0000) 2024-04-25T11:56:01.819 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T11:56:01.858 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:56:01.859 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T11:56:01.860 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T11:56:01.860 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.11 2024-04-25T11:56:02.153 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:01 smithi111 systemd[1]: Stopping Ceph osd.11 for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:56:02.153 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:02 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:02.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:56:02.153 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:02 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:02.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:56:02.153 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:02 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:02.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:56:02.153 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:02 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:02.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:56:02.154 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:02 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:02.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:56:02.154 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:02 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:02.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:56:02.154 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:02 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:02.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-25T11:55:30.830973+0000 front 2024-04-25T11:55:30.830953+0000 (oldest deadline 2024-04-25T11:55:56.730926+0000) 2024-04-25T11:56:02.154 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:02 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:02.161+0000 7fbe6bde9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:56:02.154 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:02 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:02.161+0000 7fbe6bde9700 -1 osd.11 102 *** Got signal Terminated *** 2024-04-25T11:56:02.154 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:02 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:02.161+0000 7fbe6bde9700 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T11:56:03.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:03 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:03.134+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:56:03.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:03 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:03.134+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:56:03.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:03 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:03.134+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:56:03.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:03 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:03.134+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:56:03.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:03 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:03.134+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:56:03.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:03 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:03.134+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:56:03.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:03 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:03.134+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-25T11:55:30.830973+0000 front 2024-04-25T11:55:30.830953+0000 (oldest deadline 2024-04-25T11:55:56.730926+0000) 2024-04-25T11:56:04.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:04 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:04.166+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:56:04.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:04 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:04.166+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:56:04.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:04 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:04.166+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:56:04.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:04 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:04.166+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:56:04.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:04 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:04.166+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:56:04.577 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:04 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:04.166+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:56:04.577 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:04 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:04.166+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-25T11:55:30.830973+0000 front 2024-04-25T11:55:30.830953+0000 (oldest deadline 2024-04-25T11:55:56.730926+0000) 2024-04-25T11:56:05.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:05 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:05.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:56:05.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:05 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:05.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:56:05.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:05 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:05.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:56:05.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:05 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:05.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:56:05.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:05 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:05.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:56:05.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:05 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:05.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:56:05.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:05 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:05.126+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-25T11:55:30.830973+0000 front 2024-04-25T11:55:30.830953+0000 (oldest deadline 2024-04-25T11:55:56.730926+0000) 2024-04-25T11:56:06.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:06 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:06.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:56:06.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:06 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:06.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:56:06.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:06 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:06.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:56:06.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:06 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:06.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:56:06.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:06 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:06.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:56:06.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:06 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:06.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:56:06.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:06 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:06.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-25T11:55:30.830973+0000 front 2024-04-25T11:55:30.830953+0000 (oldest deadline 2024-04-25T11:55:56.730926+0000) 2024-04-25T11:56:06.576 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:06 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:06.123+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6820 osd.8 since back 2024-04-25T11:55:43.132018+0000 front 2024-04-25T11:55:43.132060+0000 (oldest deadline 2024-04-25T11:56:05.431988+0000) 2024-04-25T11:56:07.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:07 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:07.077+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-25T11:54:39.324469+0000 front 2024-04-25T11:54:39.324397+0000 (oldest deadline 2024-04-25T11:55:03.424296+0000) 2024-04-25T11:56:07.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:07 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:07.077+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6804 osd.1 since back 2024-04-25T11:54:45.125103+0000 front 2024-04-25T11:54:45.125125+0000 (oldest deadline 2024-04-25T11:55:09.225218+0000) 2024-04-25T11:56:07.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:07 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:07.077+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6804 osd.2 since back 2024-04-25T11:54:55.126202+0000 front 2024-04-25T11:54:55.126186+0000 (oldest deadline 2024-04-25T11:55:18.026307+0000) 2024-04-25T11:56:07.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:07 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:07.077+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6812 osd.4 since back 2024-04-25T11:55:10.328384+0000 front 2024-04-25T11:55:10.328251+0000 (oldest deadline 2024-04-25T11:55:35.028313+0000) 2024-04-25T11:56:07.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:07 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:07.077+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6812 osd.5 since back 2024-04-25T11:55:19.129164+0000 front 2024-04-25T11:55:19.129268+0000 (oldest deadline 2024-04-25T11:55:42.029325+0000) 2024-04-25T11:56:07.326 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:07 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:07.077+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6822 osd.6 since back 2024-04-25T11:55:27.330644+0000 front 2024-04-25T11:55:27.330586+0000 (oldest deadline 2024-04-25T11:55:50.830426+0000) 2024-04-25T11:56:07.327 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:07 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:07.077+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.94:6820 osd.7 since back 2024-04-25T11:55:30.830973+0000 front 2024-04-25T11:55:30.830953+0000 (oldest deadline 2024-04-25T11:55:56.730926+0000) 2024-04-25T11:56:07.327 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:07 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11[58064]: 2024-04-25T11:56:07.077+0000 7fbe683dd700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.111:6820 osd.8 since back 2024-04-25T11:55:43.132018+0000 front 2024-04-25T11:55:43.132060+0000 (oldest deadline 2024-04-25T11:56:05.431988+0000) 2024-04-25T11:56:07.870 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:07 smithi111 podman[70103]: 2024-04-25 11:56:07.606754597 +0000 UTC m=+5.545465655 container died d56a4b43abfd591c47e8c4ec60c43f8e56a5dc9eb1767545225e79f36a9d1db0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T11:56:08.245 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:07 smithi111 podman[70103]: 2024-04-25 11:56:07.995159813 +0000 UTC m=+5.933870866 container cleanup d56a4b43abfd591c47e8c4ec60c43f8e56a5dc9eb1767545225e79f36a9d1db0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T11:56:08.245 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:07 smithi111 bash[70103]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11 2024-04-25T11:56:08.575 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:08 smithi111 podman[70120]: 2024-04-25 11:56:08.254708009 +0000 UTC m=+0.646082391 container remove d56a4b43abfd591c47e8c4ec60c43f8e56a5dc9eb1767545225e79f36a9d1db0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:56:08.881 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:08 smithi111 podman[70216]: 2024-04-25 11:56:08.510453435 +0000 UTC m=+0.017688929 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T11:56:08.881 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:08 smithi111 podman[70216]: 2024-04-25 11:56:08.6222353 +0000 UTC m=+0.129470783 container create 316d5f1be4e41a6693a32647b22ef083864c302caa58880c2de9fb4c3cd4552c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:56:08.882 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:08 smithi111 podman[70216]: 2024-04-25 11:56:08.89130966 +0000 UTC m=+0.398545137 container init 316d5f1be4e41a6693a32647b22ef083864c302caa58880c2de9fb4c3cd4552c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:56:09.179 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:08 smithi111 podman[70216]: 2024-04-25 11:56:08.894284514 +0000 UTC m=+0.401519999 container start 316d5f1be4e41a6693a32647b22ef083864c302caa58880c2de9fb4c3cd4552c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T11:56:09.180 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:08 smithi111 podman[70216]: 2024-04-25 11:56:08.974609747 +0000 UTC m=+0.481845237 container attach 316d5f1be4e41a6693a32647b22ef083864c302caa58880c2de9fb4c3cd4552c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T11:56:09.476 INFO:journalctl@ceph.osd.11.smithi111.stdout:Apr 25 11:56:09 smithi111 podman[70216]: 2024-04-25 11:56:09.189682131 +0000 UTC m=+0.696917607 container died 316d5f1be4e41a6693a32647b22ef083864c302caa58880c2de9fb4c3cd4552c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T11:56:09.791 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T11:56:09.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:56:09.827 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T11:56:09.827 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T11:56:09.827 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.a 2024-04-25T11:56:10.151 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:56:09 smithi083 systemd[1]: Stopping Ceph mds.a for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:56:10.409 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:56:10 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-a[71742]: 2024-04-25T11:56:10.159+0000 7faa932d6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:56:10.410 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:56:10 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-a[71742]: 2024-04-25T11:56:10.159+0000 7faa932d6700 -1 mds.a *** got signal Terminated *** 2024-04-25T11:56:12.367 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:56:12 smithi083 podman[88868]: 2024-04-25 11:56:12.105595413 +0000 UTC m=+2.068883962 container died 6680a114ecc089f49c922d24ba646d89ec25297b2c58c79caabd052b0628bf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T11:56:12.660 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:56:12 smithi083 podman[88868]: 2024-04-25 11:56:12.459460972 +0000 UTC m=+2.422749520 container cleanup 6680a114ecc089f49c922d24ba646d89ec25297b2c58c79caabd052b0628bf19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:56:12.660 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 25 11:56:12 smithi083 bash[88868]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-a 2024-04-25T11:56:12.874 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T11:56:12.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:56:12.909 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T11:56:12.910 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T11:56:12.910 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.d 2024-04-25T11:56:13.410 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 25 11:56:12 smithi083 systemd[1]: Stopping Ceph mds.d for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:56:13.410 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 25 11:56:13 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-d[70763]: 2024-04-25T11:56:13.237+0000 7fbaa27a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:56:13.410 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 25 11:56:13 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-d[70763]: 2024-04-25T11:56:13.237+0000 7fbaa27a2700 -1 mds.d *** got signal Terminated *** 2024-04-25T11:56:17.504 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 25 11:56:17 smithi083 podman[88988]: 2024-04-25 11:56:17.245533132 +0000 UTC m=+4.112686550 container died 5b29031497ff8b7f075f31363679ccd5a2b394f2c01e0b5ad110a8effad95416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T11:56:17.879 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 25 11:56:17 smithi083 podman[88988]: 2024-04-25 11:56:17.596470591 +0000 UTC m=+4.463624006 container cleanup 5b29031497ff8b7f075f31363679ccd5a2b394f2c01e0b5ad110a8effad95416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-04-25T11:56:17.880 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 25 11:56:17 smithi083 bash[88988]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-d 2024-04-25T11:56:18.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T11:56:18.050 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:56:18.050 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T11:56:18.050 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T11:56:18.051 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.g 2024-04-25T11:56:18.159 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:56:18 smithi083 systemd[1]: Stopping Ceph mds.g for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:56:18.659 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:56:18 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-g[71378]: 2024-04-25T11:56:18.382+0000 7fda550f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:56:18.660 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:56:18 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-g[71378]: 2024-04-25T11:56:18.382+0000 7fda550f7700 -1 mds.g *** got signal Terminated *** 2024-04-25T11:56:20.383 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:56:20 smithi083 podman[89110]: 2024-04-25 11:56:20.110075209 +0000 UTC m=+1.839047818 container died 1a1f96ec98c9050cc5365245aa96e09c5b071dd467517483841ea0dc8e1d94ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T11:56:20.660 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:56:20 smithi083 podman[89110]: 2024-04-25 11:56:20.517089884 +0000 UTC m=+2.246062490 container cleanup 1a1f96ec98c9050cc5365245aa96e09c5b071dd467517483841ea0dc8e1d94ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T11:56:20.660 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 25 11:56:20 smithi083 bash[89110]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-g 2024-04-25T11:56:20.882 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T11:56:20.912 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:56:20.912 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T11:56:20.913 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T11:56:20.913 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.j 2024-04-25T11:56:21.159 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 25 11:56:21 smithi083 systemd[1]: Stopping Ceph mds.j for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:56:21.659 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 25 11:56:21 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-j[70410]: 2024-04-25T11:56:21.262+0000 7f4002fa6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:56:21.660 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 25 11:56:21 smithi083 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-j[70410]: 2024-04-25T11:56:21.262+0000 7f4002fa6700 -1 mds.j *** got signal Terminated *** 2024-04-25T11:56:23.612 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 25 11:56:23 smithi083 podman[89229]: 2024-04-25 11:56:23.353225783 +0000 UTC m=+2.218275571 container died fb95b994b5c6c369fca2d4e8c0949f03d1c82deb3455d8865269dd86f3dc02b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T11:56:23.910 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 25 11:56:23 smithi083 podman[89229]: 2024-04-25 11:56:23.704390586 +0000 UTC m=+2.569440376 container cleanup fb95b994b5c6c369fca2d4e8c0949f03d1c82deb3455d8865269dd86f3dc02b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T11:56:23.910 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 25 11:56:23 smithi083 bash[89229]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-j 2024-04-25T11:56:24.113 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T11:56:24.144 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:56:24.144 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T11:56:24.144 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T11:56:24.144 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.b 2024-04-25T11:56:24.447 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:56:24 smithi094 systemd[1]: Stopping Ceph mds.b for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:56:24.706 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:56:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b[63285]: 2024-04-25T11:56:24.455+0000 7f7dd295e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:56:24.706 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:56:24 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b[63285]: 2024-04-25T11:56:24.455+0000 7f7dd295e700 -1 mds.b *** got signal Terminated *** 2024-04-25T11:56:28.488 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:56:28 smithi094 podman[69831]: 2024-04-25 11:56:28.12511496 +0000 UTC m=+3.772477482 container died abf805b5081f4b5f242f708566620d79713abbe5b0948d3c65f31f8a419cc943 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T11:56:28.777 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:56:28 smithi094 podman[69831]: 2024-04-25 11:56:28.597101664 +0000 UTC m=+4.244464235 container cleanup abf805b5081f4b5f242f708566620d79713abbe5b0948d3c65f31f8a419cc943 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:56:28.777 INFO:journalctl@ceph.mds.b.smithi094.stdout:Apr 25 11:56:28 smithi094 bash[69831]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-b 2024-04-25T11:56:29.009 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T11:56:29.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:56:29.044 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T11:56:29.044 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T11:56:29.044 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.e 2024-04-25T11:56:29.277 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:56:29 smithi094 systemd[1]: Stopping Ceph mds.e for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:56:29.777 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:56:29 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-e[62653]: 2024-04-25T11:56:29.383+0000 7f3f80ec6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:56:29.777 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:56:29 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-e[62653]: 2024-04-25T11:56:29.383+0000 7f3f80ec6700 -1 mds.e *** got signal Terminated *** 2024-04-25T11:56:32.425 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:56:32 smithi094 podman[69964]: 2024-04-25 11:56:32.119490564 +0000 UTC m=+2.850366770 container died 5e04a085005824a06e992c92cb7260d786e5aa08213de0bcf6c1bcbaef1dbd0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:56:32.742 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:56:32 smithi094 podman[69964]: 2024-04-25 11:56:32.542484223 +0000 UTC m=+3.273360427 container cleanup 5e04a085005824a06e992c92cb7260d786e5aa08213de0bcf6c1bcbaef1dbd0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:56:32.742 INFO:journalctl@ceph.mds.e.smithi094.stdout:Apr 25 11:56:32 smithi094 bash[69964]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-e 2024-04-25T11:56:32.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T11:56:32.991 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:56:32.991 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T11:56:32.991 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T11:56:32.991 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.h 2024-04-25T11:56:33.312 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:56:33 smithi094 systemd[1]: Stopping Ceph mds.h for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:56:33.777 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:56:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-h[62961]: 2024-04-25T11:56:33.320+0000 7fe1c7929700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:56:33.777 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:56:33 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-h[62961]: 2024-04-25T11:56:33.320+0000 7fe1c7929700 -1 mds.h *** got signal Terminated *** 2024-04-25T11:56:36.378 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:56:36 smithi094 podman[70084]: 2024-04-25 11:56:36.115028423 +0000 UTC m=+2.899712855 container died 0ad77a914ff4be827de2e7762841217e343fdb7340c5203859661724fbe0d514 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T11:56:36.746 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:56:36 smithi094 podman[70084]: 2024-04-25 11:56:36.503881567 +0000 UTC m=+3.288565999 container cleanup 0ad77a914ff4be827de2e7762841217e343fdb7340c5203859661724fbe0d514 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T11:56:36.747 INFO:journalctl@ceph.mds.h.smithi094.stdout:Apr 25 11:56:36 smithi094 bash[70084]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-h 2024-04-25T11:56:36.916 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T11:56:36.947 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:56:36.947 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T11:56:36.948 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T11:56:36.948 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.k 2024-04-25T11:56:37.257 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:56:37 smithi094 systemd[1]: Stopping Ceph mds.k for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:56:37.527 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:56:37 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-k[62329]: 2024-04-25T11:56:37.265+0000 7f5c6c9bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:56:37.527 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:56:37 smithi094 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-k[62329]: 2024-04-25T11:56:37.265+0000 7f5c6c9bc700 -1 mds.k *** got signal Terminated *** 2024-04-25T11:56:40.449 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:56:40 smithi094 podman[70206]: 2024-04-25 11:56:40.112825466 +0000 UTC m=+2.943107388 container died be554274b8aea69ed308fc9ea926c5678e17cd68abfbdc9d37d5672604693fdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-k, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T11:56:40.724 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:56:40 smithi094 podman[70206]: 2024-04-25 11:56:40.5493329 +0000 UTC m=+3.379614821 container cleanup be554274b8aea69ed308fc9ea926c5678e17cd68abfbdc9d37d5672604693fdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T11:56:40.725 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:56:40 smithi094 bash[70206]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-k 2024-04-25T11:56:40.966 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T11:56:40.986 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:56:40 smithi094 podman[70220]: 2024-04-25 11:56:40.733362436 +0000 UTC m=+0.615630865 container remove be554274b8aea69ed308fc9ea926c5678e17cd68abfbdc9d37d5672604693fdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:56:40.986 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:56:40 smithi094 systemd[1]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.k.service: Deactivated successfully. 2024-04-25T11:56:40.986 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:56:40 smithi094 systemd[1]: Stopped Ceph mds.k for 7fcf54c6-02f7-11ef-bc93-c7b262605968. 2024-04-25T11:56:40.986 INFO:journalctl@ceph.mds.k.smithi094.stdout:Apr 25 11:56:40 smithi094 systemd[1]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.k.service: Consumed 2.032s CPU time. 2024-04-25T11:56:40.997 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:56:40.997 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T11:56:40.997 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T11:56:40.997 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.c 2024-04-25T11:56:41.285 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:56:41 smithi111 systemd[1]: Stopping Ceph mds.c for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:56:41.545 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:56:41 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-c[63499]: 2024-04-25T11:56:41.293+0000 7faca9695700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:56:41.545 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:56:41 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-c[63499]: 2024-04-25T11:56:41.293+0000 7faca9695700 -1 mds.c *** got signal Terminated *** 2024-04-25T11:56:44.403 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:56:44 smithi111 podman[70324]: 2024-04-25 11:56:44.128056329 +0000 UTC m=+2.932450709 container died b3c289675af19f5c42f96219dafb90e4828c44566e695d8d717180d6da7f990c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T11:56:44.744 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:56:44 smithi111 podman[70324]: 2024-04-25 11:56:44.502040108 +0000 UTC m=+3.306434493 container cleanup b3c289675af19f5c42f96219dafb90e4828c44566e695d8d717180d6da7f990c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T11:56:44.744 INFO:journalctl@ceph.mds.c.smithi111.stdout:Apr 25 11:56:44 smithi111 bash[70324]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-c 2024-04-25T11:56:44.914 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T11:56:44.948 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:56:44.949 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T11:56:44.949 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T11:56:44.949 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.f 2024-04-25T11:56:45.327 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:56:45 smithi111 systemd[1]: Stopping Ceph mds.f for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:56:45.327 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:56:45 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-f[63191]: 2024-04-25T11:56:45.263+0000 7fcaa991f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:56:45.327 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:56:45 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-f[63191]: 2024-04-25T11:56:45.263+0000 7fcaa991f700 -1 mds.f *** got signal Terminated *** 2024-04-25T11:56:48.406 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:56:48 smithi111 podman[70457]: 2024-04-25 11:56:48.125494658 +0000 UTC m=+2.951319755 container died 16d4d3696cf6bf1b3b0ff072a515be54aab2eaabd606c3ea9709576479f1dd72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T11:56:48.748 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:56:48 smithi111 podman[70457]: 2024-04-25 11:56:48.514206213 +0000 UTC m=+3.340031363 container cleanup 16d4d3696cf6bf1b3b0ff072a515be54aab2eaabd606c3ea9709576479f1dd72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T11:56:48.748 INFO:journalctl@ceph.mds.f.smithi111.stdout:Apr 25 11:56:48 smithi111 bash[70457]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-f 2024-04-25T11:56:48.874 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T11:56:48.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:56:48.906 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T11:56:48.906 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T11:56:48.906 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.i 2024-04-25T11:56:49.076 INFO:journalctl@ceph.mds.i.smithi111.stdout:Apr 25 11:56:48 smithi111 systemd[1]: Stopping Ceph mds.i for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:56:49.576 INFO:journalctl@ceph.mds.i.smithi111.stdout:Apr 25 11:56:49 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-i[63813]: 2024-04-25T11:56:49.234+0000 7f4c3c13a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:56:49.576 INFO:journalctl@ceph.mds.i.smithi111.stdout:Apr 25 11:56:49 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-i[63813]: 2024-04-25T11:56:49.234+0000 7f4c3c13a700 -1 mds.i *** got signal Terminated *** 2024-04-25T11:56:52.326 INFO:journalctl@ceph.mds.i.smithi111.stdout:Apr 25 11:56:52 smithi111 podman[70579]: 2024-04-25 11:56:52.126199979 +0000 UTC m=+2.997692744 container died ab2aba19559da92b1a0066785ab5a52142cb66bbff737294b5f98c8197453451 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T11:56:52.826 INFO:journalctl@ceph.mds.i.smithi111.stdout:Apr 25 11:56:52 smithi111 podman[70579]: 2024-04-25 11:56:52.584405307 +0000 UTC m=+3.455898079 container cleanup ab2aba19559da92b1a0066785ab5a52142cb66bbff737294b5f98c8197453451 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T11:56:52.827 INFO:journalctl@ceph.mds.i.smithi111.stdout:Apr 25 11:56:52 smithi111 bash[70579]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-i 2024-04-25T11:56:52.985 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T11:56:53.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:56:53.016 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T11:56:53.017 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T11:56:53.017 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.l 2024-04-25T11:56:53.321 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:56:53 smithi111 systemd[1]: Stopping Ceph mds.l for 7fcf54c6-02f7-11ef-bc93-c7b262605968... 2024-04-25T11:56:53.576 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:56:53 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-l[62887]: 2024-04-25T11:56:53.329+0000 7fbfb9901700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T11:56:53.577 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:56:53 smithi111 ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-l[62887]: 2024-04-25T11:56:53.329+0000 7fbfb9901700 -1 mds.l *** got signal Terminated *** 2024-04-25T11:56:56.480 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:56:56 smithi111 podman[70701]: 2024-04-25 11:56:56.127016609 +0000 UTC m=+2.889501141 container died ee8cf3d49253b6dfbe6acbc0500c7acd733196cc2e03c2f287cafec673fc269b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T11:56:56.747 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:56:56 smithi111 podman[70701]: 2024-04-25 11:56:56.546532148 +0000 UTC m=+3.309016678 container cleanup ee8cf3d49253b6dfbe6acbc0500c7acd733196cc2e03c2f287cafec673fc269b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T11:56:56.747 INFO:journalctl@ceph.mds.l.smithi111.stdout:Apr 25 11:56:56 smithi111 bash[70701]: ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968-mds-l 2024-04-25T11:56:56.960 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7fcf54c6-02f7-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T11:56:56.990 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T11:56:56.990 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T11:56:56.990 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T11:56:57.177 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:56:59.730 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T11:56:59.919 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:57:02.351 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T11:57:02.540 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:57:04.941 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:57:04.973 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:57:05.004 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T11:57:05.029 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T11:57:05.032 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327/remote/smithi083/crash 2024-04-25T11:57:05.033 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/crash -- . 2024-04-25T11:57:05.099 INFO:teuthology.orchestra.run.smithi083.stderr:tar: /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T11:57:05.099 INFO:teuthology.orchestra.run.smithi083.stderr:tar: Error is not recoverable: exiting now 2024-04-25T11:57:05.101 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327/remote/smithi094/crash 2024-04-25T11:57:05.102 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/crash -- . 2024-04-25T11:57:05.212 INFO:teuthology.orchestra.run.smithi094.stderr:tar: /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T11:57:05.212 INFO:teuthology.orchestra.run.smithi094.stderr:tar: Error is not recoverable: exiting now 2024-04-25T11:57:05.214 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327/remote/smithi111/crash 2024-04-25T11:57:05.215 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/crash -- . 2024-04-25T11:57:05.244 INFO:teuthology.orchestra.run.smithi111.stderr:tar: /var/lib/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T11:57:05.244 INFO:teuthology.orchestra.run.smithi111.stderr:tar: Error is not recoverable: exiting now 2024-04-25T11:57:05.246 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T11:57:05.246 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T11:57:05.284 INFO:tasks.cephadm:Compressing logs... 2024-04-25T11:57:05.285 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:57:05.327 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:57:05.330 DEBUG:teuthology.orchestra.run.smithi111:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:57:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T11:57:05.351 INFO:teuthology.orchestra.run.smithi083.stderr:: No such file or directory 2024-04-25T11:57:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.log 2024-04-25T11:57:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T11:57:05.352 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T11:57:05.362 INFO:teuthology.orchestra.run.smithi111.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T11:57:05.362 INFO:teuthology.orchestra.run.smithi111.stderr:: No such file or directory 2024-04-25T11:57:05.362 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.log 2024-04-25T11:57:05.362 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T11:57:05.363 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T11:57:05.363 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T11:57:05.363 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.2.log: /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T11:57:05.363 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T11:57:05.363 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T11:57:05.363 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.c.log: /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mon.c.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T11:57:05.363 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T11:57:05.363 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T11:57:05.364 INFO:teuthology.orchestra.run.smithi111.stderr: 87.8% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T11:57:05.364 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T11:57:05.364 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.5% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T11:57:05.366 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T11:57:05.366 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T11:57:05.367 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.g.log: /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T11:57:05.367 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T11:57:05.367 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T11:57:05.367 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mgr.x.log: 87.8% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T11:57:05.367 INFO:teuthology.orchestra.run.smithi083.stderr: 90.3% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T11:57:05.367 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T11:57:05.367 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T11:57:05.367 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.6.log: 94.6% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T11:57:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T11:57:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T11:57:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T11:57:05.368 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.a.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T11:57:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T11:57:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:: No such file or directory 2024-04-25T11:57:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.log 2024-04-25T11:57:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T11:57:05.371 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T11:57:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T11:57:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T11:57:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.7.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T11:57:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T11:57:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T11:57:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T11:57:05.372 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip 87.8% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T11:57:05.372 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T11:57:05.372 INFO:teuthology.orchestra.run.smithi094.stderr: 90.5% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T11:57:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T11:57:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.h.log: /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T11:57:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T11:57:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T11:57:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.e.log: 91.8% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T11:57:05.373 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T11:57:05.377 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T11:57:05.377 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T11:57:05.377 INFO:teuthology.orchestra.run.smithi111.stderr: 91.9% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T11:57:05.377 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.4% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T11:57:05.378 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T11:57:05.378 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T11:57:05.378 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T11:57:05.381 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T11:57:05.382 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T11:57:05.384 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T11:57:05.387 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T11:57:05.390 INFO:teuthology.orchestra.run.smithi083.stderr: 94.5% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T11:57:05.410 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.l.log: 93.1% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T11:57:05.410 INFO:teuthology.orchestra.run.smithi111.stderr: 91.0% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T11:57:05.413 INFO:teuthology.orchestra.run.smithi111.stderr: 93.1% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T11:57:05.415 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T11:57:05.418 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.k.log: 93.2% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T11:57:05.423 INFO:teuthology.orchestra.run.smithi083.stderr: 94.4% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T11:57:05.426 INFO:teuthology.orchestra.run.smithi111.stderr: 92.9% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T11:57:05.447 INFO:teuthology.orchestra.run.smithi094.stderr: 94.6% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T11:57:05.456 INFO:teuthology.orchestra.run.smithi094.stderr: 94.1% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T11:57:05.462 INFO:teuthology.orchestra.run.smithi111.stderr: 94.4% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T11:57:05.572 INFO:teuthology.orchestra.run.smithi094.stderr: 93.3% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T11:57:05.608 INFO:teuthology.orchestra.run.smithi083.stderr: 90.3% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T11:57:05.657 INFO:teuthology.orchestra.run.smithi111.stderr: 92.3% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T11:57:05.680 INFO:teuthology.orchestra.run.smithi094.stderr: 92.2% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T11:57:05.803 INFO:teuthology.orchestra.run.smithi111.stderr: 92.9% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T11:57:05.816 INFO:teuthology.orchestra.run.smithi094.stderr: 92.5% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T11:57:05.840 INFO:teuthology.orchestra.run.smithi111.stderr: 92.7% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T11:57:05.853 INFO:teuthology.orchestra.run.smithi083.stderr: 92.9% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T11:57:05.856 INFO:teuthology.orchestra.run.smithi111.stderr: 92.7% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T11:57:05.871 INFO:teuthology.orchestra.run.smithi094.stderr: 92.9% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T11:57:05.910 INFO:teuthology.orchestra.run.smithi083.stderr: 92.6% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T11:57:05.964 INFO:teuthology.orchestra.run.smithi083.stderr: 92.8% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T11:57:06.000 INFO:teuthology.orchestra.run.smithi094.stderr: 92.8% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T11:57:06.007 INFO:teuthology.orchestra.run.smithi083.stderr: 92.9% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T11:57:06.118 INFO:teuthology.orchestra.run.smithi111.stderr: 92.8% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T11:57:06.119 INFO:teuthology.orchestra.run.smithi111.stderr: 2024-04-25T11:57:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:real 0m0.776s 2024-04-25T11:57:06.119 INFO:teuthology.orchestra.run.smithi111.stderr:user 0m2.643s 2024-04-25T11:57:06.120 INFO:teuthology.orchestra.run.smithi111.stderr:sys 0m0.111s 2024-04-25T11:57:06.182 INFO:teuthology.orchestra.run.smithi094.stderr: 93.0% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T11:57:06.185 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-25T11:57:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m0.831s 2024-04-25T11:57:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:user 0m3.060s 2024-04-25T11:57:06.186 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m0.123s 2024-04-25T11:57:06.256 INFO:teuthology.orchestra.run.smithi083.stderr: 90.6% -- replaced with /var/log/ceph/7fcf54c6-02f7-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T11:57:06.258 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-25T11:57:06.258 INFO:teuthology.orchestra.run.smithi083.stderr:real 0m0.918s 2024-04-25T11:57:06.258 INFO:teuthology.orchestra.run.smithi083.stderr:user 0m3.488s 2024-04-25T11:57:06.258 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m0.137s 2024-04-25T11:57:06.260 INFO:tasks.cephadm:Archiving logs... 2024-04-25T11:57:06.260 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327/remote/smithi083/log 2024-04-25T11:57:06.368 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:57:07.556 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327/remote/smithi094/log 2024-04-25T11:57:07.557 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:57:07.999 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327/remote/smithi111/log 2024-04-25T11:57:07.999 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:57:08.415 INFO:tasks.cephadm:Removing cluster... 2024-04-25T11:57:08.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 --force 2024-04-25T11:57:08.595 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:57:08.976 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 --force 2024-04-25T11:57:09.166 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:57:09.466 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7fcf54c6-02f7-11ef-bc93-c7b262605968 --force 2024-04-25T11:57:09.655 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 7fcf54c6-02f7-11ef-bc93-c7b262605968 2024-04-25T11:57:09.923 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T11:57:09.923 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T11:57:09.942 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T11:57:09.957 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T11:57:09.972 INFO:tasks.cephadm:Teardown complete 2024-04-25T11:57:09.972 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T11:57:09.984 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T11:57:09.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:57:09.991 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:57:09.999 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:57:10.107 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T11:57:10.118 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T11:57:10.119 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:57:10.122 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:57:10.124 DEBUG:teuthology.orchestra.run.smithi111:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:57:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:57:10.136 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:57:10.138 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:57:10.140 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:57:10.140 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-04-25T11:57:10.140 INFO:teuthology.orchestra.run.smithi094.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +720us[ +707us] +/- 54ms 2024-04-25T11:57:10.140 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 49 -617us[ -617us] +/- 63ms 2024-04-25T11:57:10.140 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -208us[ -221us] +/- 70ms 2024-04-25T11:57:10.140 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:57:10.142 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:57:10.142 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-04-25T11:57:10.142 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv01.front.sepia.ceph.com 3 6 377 49 +715us[ +701us] +/- 54ms 2024-04-25T11:57:10.142 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 48 -628us[ -628us] +/- 63ms 2024-04-25T11:57:10.142 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 -221us[ -221us] +/- 71ms 2024-04-25T11:57:10.143 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:57:10.143 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:57:10.144 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-04-25T11:57:10.144 INFO:teuthology.orchestra.run.smithi111.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 +718us[ +711us] +/- 54ms 2024-04-25T11:57:10.144 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -609us[ -616us] +/- 63ms 2024-04-25T11:57:10.144 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 -218us[ -225us] +/- 70ms 2024-04-25T11:57:10.144 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:57:10.144 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T11:57:10.157 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T11:57:10.158 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T11:57:10.168 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:57:10.260 DEBUG:teuthology.orchestra.run.smithi094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:57:10.325 DEBUG:teuthology.orchestra.run.smithi111:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:57:10.388 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:57:10.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:57:10.417 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:57:10.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:57:10.443 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:57:10.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:57:10.470 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T11:57:10.481 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T11:57:10.492 INFO:teuthology.task.internal:Duration was 1835.536434 seconds 2024-04-25T11:57:10.492 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T11:57:10.502 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T11:57:10.502 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:57:10.505 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:57:10.507 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:57:10.541 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:57:10.541 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:57:10.545 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:57:11.050 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T11:57:11.050 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-04-25T11:57:11.050 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:57:11.071 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T11:57:11.072 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:57:11.116 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi111.front.sepia.ceph.com 2024-04-25T11:57:11.117 DEBUG:teuthology.orchestra.run.smithi111:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:57:11.142 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T11:57:11.143 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:57:11.145 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:57:11.159 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:57:11.208 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T11:57:11.208 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:57:11.212 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:57:11.226 DEBUG:teuthology.orchestra.run.smithi111:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:57:11.587 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T11:57:11.610 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T11:57:11.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:57:11.634 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:57:11.659 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:57:11.682 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T11:57:11.712 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:57:11.718 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:57:11.724 DEBUG:teuthology.orchestra.run.smithi111:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:57:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-04-25T11:57:11.743 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-04-25T11:57:11.750 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = core 2024-04-25T11:57:11.778 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:57:11.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:57:11.809 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:57:11.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:57:11.837 DEBUG:teuthology.orchestra.run.smithi111:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:57:11.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:57:11.853 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T11:57:11.892 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T11:57:11.892 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327/remote/smithi083 2024-04-25T11:57:11.893 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:57:12.057 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327/remote/smithi094 2024-04-25T11:57:12.058 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:57:12.118 DEBUG:teuthology.misc:Transferring archived files from smithi111:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327/remote/smithi111 2024-04-25T11:57:12.118 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:57:12.160 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T11:57:12.160 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:57:12.163 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:57:12.165 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:57:12.209 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T11:57:12.223 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T11:57:12.223 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T11:57:12.255 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T11:57:12.255 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:57:12.258 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:57:12.260 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:57:12.269 INFO:teuthology.orchestra.run.smithi083.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:57 /home/ubuntu/cephtest 2024-04-25T11:57:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:57 /home/ubuntu/cephtest 2024-04-25T11:57:12.274 INFO:teuthology.orchestra.run.smithi111.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:57 /home/ubuntu/cephtest 2024-04-25T11:57:12.275 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T11:57:12.287 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T11:57:12.312 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T11:57:12.338 DEBUG:teuthology.nuke:shortname: smithi083 2024-04-25T11:57:12.338 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T11:57:12.364 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:20:29.871160', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGISUBihUgZsFxZud9ytPOEp95MyZ0z3iUwK7r2LTaPv9wcVxyoCXKiMLbb7BMbL2owZiIAfB/BhhitLQhbefh0='} 2024-04-25T11:57:12.369 DEBUG:teuthology.nuke:shortname: smithi094 2024-04-25T11:57:12.369 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T11:57:12.392 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:20:29.869416', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPo3TArzamFKHNhq3G02/mvY2LXS2S1nf+/IN1v8ExnA0UOnS1C03TIXrv4muGJVaEks6Iei29SeYmeQ4XuSd4M='} 2024-04-25T11:57:12.397 DEBUG:teuthology.nuke:shortname: smithi111 2024-04-25T11:57:12.398 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T11:57:12.419 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_11:13:36-fs-wip-lusov-quiescer-distro-default-smithi/7673327', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 11:20:29.867476', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKMmefO1xnIngfSC8rDW1pi8MU52/VOEWpehgYjOL2qp7AM6EDBE6uOSR0LEhAOLAmDpC7BbPcx2GVHaSmI305U='} 2024-04-25T11:57:12.429 INFO:teuthology.orchestra.console.smithi083:Power off 2024-04-25T11:57:12.429 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T11:57:12.452 DEBUG:teuthology.orchestra.console.smithi083:power off output: Chassis Power Control: Down/Off 2024-04-25T11:57:12.452 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:57:12.470 INFO:teuthology.orchestra.console.smithi094:Power off 2024-04-25T11:57:12.470 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T11:57:12.485 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-25T11:57:12.494 DEBUG:teuthology.orchestra.console.smithi094:power off output: Chassis Power Control: Down/Off 2024-04-25T11:57:12.494 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:57:12.513 INFO:teuthology.orchestra.console.smithi111:Power off 2024-04-25T11:57:12.514 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T11:57:12.524 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2024-04-25T11:57:12.537 DEBUG:teuthology.orchestra.console.smithi111:power off output: Chassis Power Control: Down/Off 2024-04-25T11:57:12.537 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:57:12.561 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2024-04-25T11:57:16.487 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:57:16.526 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:57:16.562 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:57:16.598 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-25T11:57:16.636 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2024-04-25T11:57:16.673 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2024-04-25T11:57:20.602 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:57:20.637 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:57:20.674 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:57:20.713 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2024-04-25T11:57:20.747 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is off 2024-04-25T11:57:20.784 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2024-04-25T11:57:20.815 INFO:teuthology.orchestra.console.smithi083:Power off completed 2024-04-25T11:57:20.848 INFO:teuthology.orchestra.console.smithi094:Power off completed 2024-04-25T11:57:24.786 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T11:57:24.896 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is off 2024-04-25T11:57:24.997 INFO:teuthology.orchestra.console.smithi111:Power off completed 2024-04-25T11:57:25.103 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/misc}} duration: 1835.536434173584 failure_reason: '''cluster''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=f0b1f6da84f84f1bb872e8b74a7eb21f status: fail success: false 2024-04-25T11:57:25.103 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T11:57:25.189 INFO:teuthology.run:FAIL